Systems and methods for validating and correcting automated medical image annotations

ABSTRACT

Systems and methods are disclosed for manipulating image annotations. One method includes receiving an image of an individual&#39;s anatomy; automatically determining, using a processor, one or more annotations for anatomical features identified in the image of the individual&#39;s anatomy; determining a dependency or hierarchy between at least two of the one or more annotations for anatomical features identified in the image of the individual&#39;s anatomy; and generating, based on the dependency or hierarchy, a workflow prompting a user to manipulate the one or more annotations for anatomical features identified in the image of the individual&#39;s anatomy.

RELATED APPLICATION(S)

This application claims priority to U.S. Provisional Application No.61/882,492 filed Sep. 25, 2013, the entire disclosure of which is herebyincorporated herein by reference in its entirety.

FIELD OF THE DISCLOSURE

Various embodiments of the present disclosure relate generally tocreating accurate models for computational analysis. More specifically,particular embodiments of the present disclosure relate to systems andmethods for manipulating or validating and correcting automated imageannotations.

BACKGROUND

Automated image annotation plays an increasing role in commercialsystems. In particular, the medical imaging community reliesincreasingly on the automated analysis and annotation of large images.Since this automated image analysis may be used to drive patient caredecisions, it is important for the automated results be validated andappropriately corrected (if necessary) by a knowledgeable user. Thus, adesire exists to guide a user through approval or correction of theautomated image analysis and annotations.

SUMMARY

According to certain aspects of the present disclosure, systems andmethods are disclosed for manipulating image annotations. One methodincludes: Systems and methods are disclosed for manipulating imageannotations. One method includes receiving an image of an individual'sanatomy; automatically determining, using a processor, one or moreannotations for anatomical features identified in the image of theindividual's anatomy; determining a dependency or hierarchy between atleast two of the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy; and generating,based on the dependency or hierarchy, a workflow prompting a user tomanipulate the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy.

In accordance with another embodiment, a system for manipulating imageannotations comprises: a data storage device storing instructionsmanipulating image annotations; and a processor configured for:receiving an image of an individual's anatomy; automaticallydetermining, using a processor, one or more annotations for anatomicalfeatures identified in the image of the individual's anatomy;determining a dependency or hierarchy between at least two of the one ormore annotations for anatomical features identified in the image of theindividual's anatomy; and generating, based on the dependency orhierarchy, a workflow prompting a user to manipulate the one or moreannotations for anatomical features identified in the image of theindividual's anatomy.

In accordance with yet another embodiment, a non-transitory computerreadable medium for use on a computer system containingcomputer-executable programming instructions manipulating imageannotations is provided. The method includes: receiving an image of anindividual's anatomy; automatically determining, using a processor, oneor more annotations for anatomical features identified in the image ofthe individual's anatomy; determining a dependency or hierarchy betweenat least two of the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy; and generating,based on the dependency or hierarchy, a workflow prompting a user tomanipulate the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of an exemplary system and network formanipulating image annotations, especially in validating and correctingautomated image annotations, according to an exemplary embodiment of thepresent disclosure.

FIG. 1B is a block diagram of an exemplary method for validating andcorrecting automated image annotations, according to an exemplaryembodiment of the present disclosure.

FIG. 1C is another block diagram of an exemplary method for validatingand correcting automated image annotations, according to an exemplaryembodiment of the present disclosure.

FIG. 1D is a diagram of an exemplary series of dependencies forvalidating and correcting automated image annotations, according to anexemplary embodiment of the present disclosure.

FIG. 2 is a block diagram of an exemplary red flags task, according toan exemplary embodiment of the present disclosure.

FIG. 3A is a block diagram of an exemplary aorta task, according to anexemplary embodiment of the present disclosure.

FIG. 3B is a display of an aorta inspect mode, according to an exemplaryembodiment of the present disclosure.

FIG. 3C is a display of an aorta edit mode, according to an exemplaryembodiment of the present disclosure.

FIG. 3D is a display of an aorta create mode, according to an exemplaryembodiment of the present disclosure.

FIG. 4A is a block diagram of an exemplary landmarks task, according toan exemplary embodiment of the present disclosure.

FIG. 4B is a display of landmarks mode, according to an exemplaryembodiment of the present disclosure.

FIG. 5A is a block diagram of an exemplary myocardium task, according toan exemplary embodiment of the present disclosure.

FIG. 5B is a display of myocardium inspect mode, according to anexemplary embodiment of the present disclosure.

FIG. 5C is a display of myocardium edit mode, according to an exemplaryembodiment of the present disclosure.

FIG. 5D is a display of myocardium create mode, according to anexemplary embodiment of the present disclosure.

FIG. 6A is a block diagram of an exemplary centerlines task, accordingto an exemplary embodiment of the present disclosure.

FIG. 6B is a display of centerlines mode, according to an exemplaryembodiment of the present disclosure.

FIG. 6C is a display of centerlines labeling mode, according to anexemplary embodiment of the present disclosure.

FIG. 7A is a block diagram of an exemplary lumen task, according to anexemplary embodiment of the present disclosure.

FIG. 7B is a display of lumen mode, according to an exemplary embodimentof the present disclosure.

FIG. 8A is a block diagram of an exemplary finalize task, according toan exemplary embodiment of the present disclosure.

FIG. 8B is a display of finalize mode 820, according to an exemplaryembodiment of the present disclosure.

FIG. 9 is a display of navigate mode 900, according to an exemplaryembodiment of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

The present disclosure is directed to facilitating the creation ofaccurate models, for instance, models in preparation for computationalanalysis. Specifically, the present disclosure may include a guidedworkflow, by which users may be taken through multiple steps in order tovalidate a model. For example, an embodiment of the present disclosuremay facilitate validating a segmentation used for modeling andsimulation for blood flow of the heart (Taylor, Fonte, & Min,“Computational Fluid Dynamics Applied to Cardiac Computed Tomography forNoninvasive Quantification of Fractional Flow Reserve.” Journal of theAmerican College of Cardiology. 2013 Jun. 4; 61(22):2233-2241, thedisclosure of which is incorporated herein by reference in itsentirety). Segmentations for such modeling and simulation may involve anextremely precise image segmentation from a cardiac CT image to create apatient-specific 3D geometric model of the patient's heart. The presentdisclosure may guide users through automated image annotations, thusefficiently and effectively providing structure for trained users invalidating and correcting the patient-specific 3D model to ensurereliability of the blood flow simulations and the correctness of thetreatment decisions derived from the simulation. The process is designedto decrease analyst processing time, maintain or increase accuracy ofcomputed flow reserve (FFRct) results, and improve reproducibility ofmeasured fractional flow reserve (mFFR) results.

In other words, the method and systems for guiding a user through theimage annotations may help analysts examine segmentation results (e.g.,resultant image annotations from segmentation). The present disclosureinvolves a guided workflow for a user verifying a segmentation, for usein creating a segmentation for an image dataset. The term imageannotation may generically represent any identification or indicia in animage (including, but not limited to, a 2D image, a 3D image or an imageof greater dimension) that includes a localization or a labeling.Examples include: localization of particular points (landmarks),localization of lines and curves (e.g., diameters, centerlines), 2Dregions of interest (a 2D segmentation), 3D regions of interest (a 3Dsegmentation), an n-D region of interest (an n-D segmentation), ann-D+time region of interest (an n-D segmentation tracked through time)or a labeling/classification of one or more identified structures orsections of the image (e.g. a label or score for each image in atime-series).

The present disclosure may include several methods for designing ordirecting workflows to assist analysts in the validation, or automatingparts of the validation once there is a collective pool of analystvalidation information.

In the following disclosure, each set of annotations may be associatedwith a validation “task.” For example, aorta segmentation may be an“aorta task,” identifying ostia points may be a “landmarks task,” leftventricle myocardium segmentation may be during a “myocardium task,”vessel labeling may occur during a “centerline labeling task”, andvessel lumen segmentation may take place for a “segmentation task.” Inone embodiment, the workflow of the present disclosure may include anorder or sequence for presenting each of the tasks. For example, thetasks may be presented in an order of dependency, where annotations thatmay govern or impact other annotations, may be presented earlier in theworkflow. This way, subsequent annotations may be adjusted based onvalidations or corrections to their respective governing annotations. Inother words, if one of the annotations with a dependency is modified bya user upon review, automated algorithms for consequent annotations maybe re-run with the modified annotations as input.

In one embodiment, the tasks comprising the workflow of the presentdisclosure may include prompting review for red flags, then proceedingwith prompting users with various validation tasks including, forexample, tasks for validating an aorta, landmarks, myocardium,centerlines, centerline labeling, and segmentation. As a final task, auser may finalize the model. The dependency and ordering of the imageannotations may be described in more detail by FIG. 1D.

It should be appreciated that any type of computing system, such as acomputer having a digital memory storage device, a processor, and anydesired user interfaces may be configured to perform the presentlydisclosed methods. Moreover, any type of servers, clustered computers,and/or cloud computers may be configured to perform the presentlydisclosed methods. Any of the above referenced devices may be connectedto a network, such as the Internet, to receive and transmit data used inperforming the presently disclosed methods.

Referring now to the figures, FIG. 1A depicts a block diagram of anexemplary system and network for validating and correcting automatedimage annotations. Specifically, FIG. 1 depicts a plurality ofphysicians 102 and third party providers 104, any of whom may beconnected to an electronic network 100, such as the Internet, throughone or more computers, servers, and/or handheld mobile devices.Physicians 102 and/or third party providers 104 may create or otherwiseobtain images of, for instance, one or more patients' cardiac and/orvascular systems. The physicians 102 and/or third party providers 104may also obtain any combination of patient-specific information, such asage, medical history, blood pressure, blood viscosity, etc. Physicians102 and/or third party providers 104 may transmit the cardiac/vascularimages and/or patient-specific information to server systems 106 overthe electronic network 100. Server systems 106 may include storagedevices for storing images and data received from physicians 102 and/orthird party providers 104. Server systems 106 may also includeprocessing devices for processing images and data stored in the storagedevices.

FIG. 1B is a block diagram of an exemplary method 110 for validating andcorrecting automated image annotations, according to an exemplaryembodiment of the present disclosure. The method of FIG. 1B may beperformed by server systems 106, based on information, images, and datareceived from physicians 102 and/or third party providers 104 overelectronic network 100. In one embodiment, step 111 may includeobtaining an image. For example, step 111 may include acquiring adigital representation (e.g., the memory or digital storage [e.g., harddrive, network drive] of a computational device such as a computer,laptop, DSP, server, etc.) of image data. In one embodiment, step 113may include applying an automated image analysis system to produce a setof image annotations. Example automated image analysis systems includebut are not limited to:

-   -   a. Face detection in a digital camera    -   b. Image or video labeling (tagging) in a collection of        images/videos (e.g., YouTube)    -   c. 3D organ segmentation in CT medical images for radiation        therapy planning    -   d. 2D left ventricle segmentation in ultrasound medical images        for computing an ejection fraction    -   e. 2D cell detection in a microscopy image for image-based        cytometry    -   f. 2D cell tracking through video in an optical microscopy image        for determination of mitosis events    -   g. 3D tumor segmentation and feeder vessel centerline annotation        in CT medical images for chemoembolization planning    -   h. 3D bone fracture segmentation in CT medical images for bone        reconstruction planning    -   i. Tumor detection and segmentation in a digital mammography        application

In one embodiment, step 115 may include determining a dependency of theimage annotations on each other, if any. For example, one or more imageannotations may depend or hinge on other annotations. Put another way,one or more image annotations may affect or impact other annotations. Inone embodiment, step 117 may include presenting, to a user, eachannotation in order of dependency (e.g., an annotation is presentedearlier if another annotation is dependent upon it). In some instances,a subset of annotations may be presented, for example, based on its needfor user validation. In other words, step 117 may include refrainingfrom presenting a group of annotations. Refraining from presenting anannotation may occur in cases where an automated image analysis reportshigh confidence in the automated annotation correctness or in caseswhere the annotation is of low importance, for instance. In oneembodiment, step 117 may include determining which annotations topresent or not present. For each annotation, step 117 of presenting anannotation to a user may include providing one or more visualizations ofthe annotation and/or supporting image data that allows the user to viewand validate the annotation. The visualizations may or may not allowinteractive interrogation from the user. Presenting annotations in step117 may further include providing tools that enable the user to modifythe annotation to achieve a quality meeting the user's satisfaction.Step 117 may still further include allowing a user to accept theannotation as validated before proceeding to the subsequent annotation.In some cases, step 117 may also include detecting modifications toannotations, made in the annotation visualizations (e.g., via thetools). In such situations, the dependent annotations may be recomputed.For example, the re-computing may be performed automatically. In somecases, already validated annotations may not be subject to there-computing.

In one embodiment, step 119 may include determining one or moreannotations as being highly critical to an application. Step 119 mayfurther include representing such annotations to a user or presentingsuch annotations to one or more additional users for validation. Thedetermination of “highly critical” status may be performed in manyapplication-specific ways (e.g. including, but not limited to, aspectsof the annotation comprising, size, shape, appearance, density, spatialor other relationship to other annotations). For example, theannotations determined to be highly critical may be presented andreviewed by:

-   -   a. The user for a second time    -   b. A supervisor    -   c. Another expert    -   d. A panel of other individuals    -   e. As part of a training exercise to ensure reproducibility

Step 121 may include storing or saving validated annotations to anelectronic storage medium (e.g., hard drive, computer RAM, networkcommunication channel).

FIG. 1C is another block diagram of an exemplary method 120 forvalidating and correcting automated image annotations, according to anexemplary embodiment of the present disclosure. The method of FIG. 1Cmay be performed by server systems 106, based on information, images,and data received from physicians 102 and/or third party providers 104over electronic network 100. Method 120 may serve as a specificembodiment of method 110.

A specific embodiment of the previously described system and method mayinclude a blood flow simulation and determination of a simulatedfractional flow reserve. The present disclosure is directed to using anautomated image analysis system to generate a 3D patient-specific model(segmentation) of the vascular geometry (aorta and coronaries), alabeling of the coronary vessels, a 3D patient-specific model(segmentation) of the left ventricle myocardium, and an aortic valvepoint.

In one embodiment, generating the 3D model may begin with step 123 ofacquiring a digital representation (e.g., the memory or digital storage[e.g., hard drive, network drive] of a computational device such as acomputer, laptop, DSP, server, etc.) of a patient's heart scan (a 3Dcardiac CT image). In one embodiment, step 125 may include the automatedimage analysis system applying image annotations to the model or imagesassociated with the model. Specifically, the system may automaticallyannotate the following: a 3D segmentation of the aorta (e.g., using(Kirisli, et al., “Fully automatic cardiac segmentation from 3D CTAdata: a multi-atlas based approach,” Proceeding of SPIE. Vol. 7623,762305-9, 2010), the disclosure of which is incorporated herein byreference in its entirety), the location of ostia points (e.g., using(Zheng, et al., “Efficient Detection of Native and Bypass Coronary Ostiain Cardiac CT Volumes: Anatomical vs. Pathological Structures,” Proc.Intl Conf. Medical Image Computing and Computer Assisted Intervention,2011) the disclosure of which is incorporated herein by reference in itsentirety), the location of the aortic valve point (e.g., using (Zheng,Barbu, Georgescu, Scheuering, & Comaniciu, “Four-Chamber Heart Modelingand Automatic Segmentation for 3D Cardiac CT Volumes Using MarginalSpace Learning and Steerable Features,” IEEE Transactions on MedicalImaging, Vol. 27, No. 11, pp. 1668-1681, 2008), the disclosure of whichis incorporated herein by reference in its entirety), coronary vesselcenterlines (e.g., using (Kitamura, Li, & Ito, “Automatic coronaryextraction by supervised detection and shape matching” InternationalSymposium on Biomedical Imaging (ISBI), 2012 9th Institute of Electricaland Electronics Engineers (IEEE) International Symposium on May 2-52012), the disclosure of which is incorporated herein by reference inits entirety), labeling of the vessel centerlines by vessel name (i.e.,right coronary artery (RCA), left anterior descending artery (LAD), leftcircumflex artery (LCX), where this labeling may be performed by using aset of training labels to determine the statistics of the geometricpositioning of each labeled vessel centerline and assigning the vesselcenterline the labels having the maximum likelihood based on itsgeometric position (see e.g., (Lorenz & Berg, “A Comprehensive ShapeModel of the Heart” Medical Image Analysis, vol. 10, no. 4, pp. 657-670,(18 May 2006), the disclosure of which is incorporated herein byreference in its entirety), 3D segmentation of the coronary vessel lumen(e.g., using (Schaap, et al. “Robust Shape Regression for SupervisedVessel Segmentation and its Application to Coronary Segmentation in CTA”IEEE Transactions on Medical Imaging, Vol. 30, No. 11, November 2011),the disclosure of which is incorporated herein by reference in itsentirety), 3D segmentation of the left ventricle myocardium (e.g., using(Kirisli, et al., 2010)), etc. In other words, step 127 may includepresenting image annotations to a user in the order:

-   -   a. Aorta segmentation    -   b. Ostia points    -   c. Aortic valve point    -   d. Left ventricle myocardium segmentation    -   e. Coronary vessel centerlines    -   f. Vessel labeling    -   g. Vessel lumen segmentation

Such an order may show a dependent relationship among image annotations.If one of the annotations with a dependency is modified by a user uponreview, the automated algorithms for consequent annotations may bere-run with the modified annotation as input.

Following a completely validated segmentation, step 129 may includeanalyzing the lumen segmentation for critical regions. For example, step129 may include finding critical regions by determining areas wherediameter reduction exceeds 50%. Any lumen segmentation components thatare marked critical may be presented again to the user for a secondexamination and modification, if necessary. In one embodiment, step 131may include saving a fully validated model (including any modifications)to an electronic storage medium (e.g., hard drive, computer RAM, networkcommunication channel).

FIG. 1D is a diagram of an exemplary series of dependencies 140 forvalidating and correcting automated image annotations, according to anexemplary embodiment of the present disclosure. For example, aortasegmentation 133 (e.g., detailed in FIG. 3A showing aorta task 300) mayserve as a starting point. Subsequent validation steps may depend on theaorta segmentation 133. Following aorta segmentation 133, a workflow maypresent steps relating to an aortic valve point 135 and/or ostia points137 (e.g., corresponding to landmarks task 400 shown in FIG. 4A). Thenext steps may relate to left ventricle myocardium segmentation 143(e.g., myocardium task 500 of FIG. 5A). A subsequent dependent task mayinclude analysis of coronary vessel centerlines 139 (e.g., shown incenterlines task 600 of FIG. 6). From the workflow steps relating tocoronary vessel centerlines 139, the workflow may lead a user to vessellabeling 141 or vessel lumen segmentation 143 (e.g., lumen task 700 ofFIG. 7A). This chain of dependencies in tasks may guide a user through apredictable, logical series of validation steps towards building acomplete model.

As previously discussed, annotations may be presented via one or morevisualizations. In one embodiment, each visualization of the annotationsmay include tools to help complete the validated segmentation. Forexample, “Smart Paint” and “Nudge” tools may be used across aworkstation, for any task that utilizes a segmentation. Specificfunctionality for the different tasks or modes may differ, but theunderlying principle of the tool may be consistent across the differenttasks.

For example, Smart Paint may operate by adding “inside” (or “object”)and “outside” (or “background”) seeds to the image data. The smart paintalgorithm may generate a new segmentation using these seeds and possiblyany existing segmentation information. The smart paint algorithm may usea version of the random walker algorithm for segmentation. In somecases, the user may add a minimal number of seeds to obtain a goodsegmentation. Ultimately, with enough paint, a user may obtain anysegmentation. Smart Paint may function on the MPRs and add paint in adisc-like shape. Further, the size of a brush for Smart Paint may beadjustable. In one situation, a size of the brush may increase when auser scrolls up on a mouse wheel, and decrease in size when the userscrolls down on the mouse wheel. Furthermore, seeds may be erased (e.g.,using an Erase Seeds tool that may erase both inside and outside seeds).If a user paints over an existing seed with a seed of an opposite type,the original seed may be overwritten.

For an edit mode for an aorta task and myocardium task, Smart Paint mayupdate the segmentation with each addition or subtraction of inside oroutside seeds. The algorithm may operate using a current segmentation.The smart paint algorithm may update segmentation in a region ofinterest (ROI) around a currently added paint stroke.

For a create mode for an aorta task and myocardium task, Smart Paint mayupdate the segmentation only when prompted with a button click. In somecases, the algorithm may incorporate all paint to create a newsegmentation, rather than use the current segmentation. The algorithmmay use all paint strokes to define a ROI within which to work.

For a lumen mode, Smart Paint may affect segmentation in neighboringun-reviewed or reviewed sections. The segmentation may update with eachplacement of seeds, using the current segmentation as a starting point.The algorithm may update the segmentation in a ROI around a currentlyadded paint stroke, but incorporate all seeds contained within the ROI.

Regarding the Nudge tool, Nudge may operate by pushing segmentation in adirection of mouse movement, with a spherical shape centered at a mousecursor. If a user starts pushing a segmentation while inside asegmentation visualization, the edit may be referred to as an “InsideNudge.” If the user starts the pushing segmentation when outside thesegmentation, the edit may be referred to as an “Outside Nudge.” Thesize of the nudge may be determined by a distance between the cursor andthe segmentation. The further away the cursor, the larger the edit.Nudge may affect the segmentation directly, so this tool may work inreal time.

In one embodiment, Nudge may work in MPR views. In a further embodiment,Nudge may be available in aorta edit mode 340 (e.g., shown in FIG. 3C),myocardium edit mode 540 (e.g., shown in FIG. 5C), and lumen task 700.Functionality between the three may be very similar, with uniquevariations for the myocardium and lumen applications. For myocardiumedit mode 540, an inner and outer contour may exist since thesegmentation may only pertain to the left ventricular muscle. Thus,edits within the muscle that either push the outer contour outwards orthe inner contour inwards may be considered “Inside Nudges”; editsoutside the segmentation that either push the outer contour inwards orthe inner contour outwards, may be considered “Outside Nudges.” For thelumen task 700, in order to minimize switching sections and creatingsegmentation cusps, Nudge may work on different contours and sections.In some cases, Nudge may not operate on contours of distal sections, oron contours of branch vessels. However, Nudge may operate on contours ofthe proximal or connecting branch. This way, users may use the tools tomodify the bifurcation of both contours only on the branch section. On aparent vessel section, Nudge may affect the parent vessel section andleave a branch contour untouched.

FIG. 2 is a block diagram of an exemplary red flags task 200, accordingto an exemplary embodiment of the present disclosure. Red flags task 200may be performed by server systems 106, based on information, images,and data received from physicians 102 and/or third party providers 104over electronic network 100. In one embodiment, the purpose of red flagstask 200 may be to confirm the selection of a series of images fromwhich to create a model. For example, where a model may include modelinglumen and myocardium, red flags task 200 may be the first opportunityfor a user to review data in detail and determine adequacy of images forprocessing into a model. In addition, red flags task 200 may help usersmake note of any anatomical anomalies apparent from the series. FFRctmay include several contraindications that may prevent its use incertain patient anatomical anomalies, diseases, or in certain CTacquisition issues. As such, it is helpful to review FFRct data forappropriateness prior to processing. Red flags task 200 may methodicallypresent a user with each contraindication to verify that the patientand/or data is appropriate for processing. In some cases, a user maydetermine the appropriateness of the selection of the series forprocessing but leave an automatic determination unchanged unless aserious error in the calculation is observed.

In one embodiment, red flags task 200 may include step 201, where a usermay view and assess a series of images. For example, a user may beassessing a series of images for images that best show a lumen. In oneembodiment, step 201 may include red flags task 200 presenting a singlemulti-planar reconstruction (MPR) with constrained navigation, a userinterface (UI) element for series selection, a list of acceptable imagequality and/or patient quality elements, an exemplary image of apathology or anomaly to be assessed, and UI elements to mark particularpathology and/or anomaly as fit for acceptance or rejection. In oneembodiment, an MPR view may include image data intersected by a planethrough the image volume. More specifically, an MPR view may include anaxial MPR view (e.g., in the traverse plane). For example, the MPR viewmay be set to a default zoom and centering point; and an automaticwindow/level may be applied. In one embodiment, a window may include aview shown at a workstation visualization, including a zoom level. Alevel may include an orientation or alignment of a view. Further, theMPR view may be oriented so that the anterior direction is towards thetop of the screen. In one embodiment, an MPR view may include scrollingand windowing capabilities. In one embodiment of the red flags task 200,rotation, zoom, and panning may not be available. This is so that focusmay be kept at global red flag issues. Rotation, zoom, and panningcapabilities may be included as navigation options once a set of imagesor image annotations is already deemed acceptable for analysis and use.

In one embodiment, a user is initially presented with an axial MPR as adefault view, and a single example image of the pathology to bereviewed. The user may be tasked with reviewing image data for theparticular item. For example, step 203 of searching for red flags mayinclude reviewing image data for inadequacies. The user may be promptedor provided with tools to mark the item as acceptable or rejected,depending on whether an item presented prevents the series from beingused for processing. If a user marks an item as acceptable, the redflags task 200 may bring up the next item in the series for review. Ifan item is marked as rejected, the workstation may automatically closeand the case may be recorded as rejected. In one embodiment, a taskmanager may be displayed to present the user with further options, forinstance, the option to select another series to review, to return to anearlier item in a series, to see all available series' including a listof accepted and rejected series, etc. The user may review each item foran incorrect data list and anomaly list. As each item is marked asaccepted, a list of anomalies may be populated and marked as accepted.In one embodiment, red flags task 200 may determine that a user haseither accepted or rejected a series (step 205).

If a second series is selected for processing (i.e., different seriesare selected for lumen and myocardium processing), the second series mayalso be reviewed for appropriateness. For example, a user may beprompted to follow through step 207 of viewing and assessing an image inthe second series, step 209 of searching for red flags, and acceptingand/or rejecting the image (step 211). In one embodiment, a user may bepresented with each item sequentially. The user may then mark each itemon two image quality lists (e.g., an incorrect data list and an anomalylist). On the acceptance of the last item in the list, the next task,the aorta task, may be activated. If an item is rejected at any point,an entire series or set of annotations associated with a patient may berejected. In one embodiment, the workstation may automatically close. Inanother embodiment, the user may be presented with the next series(e.g., the first item of the next series) or a menu from which the usermay select a series. In one embodiment, red flag task 200 may only haveone mode, so mode switching options may be unnecessary.

In cases where incorrect data may be selected for processing, remainingdata sent by a sampling or test site may still be appropriate forprocessing. For example, red flags task 200 may include a determinationof whether all the series available have been analyzed (step 213). Wheremore images and/or series are available, steps 207, 209, and 211 may berepeated for images in another series. If all series have been analyzed,red flags task 200 may note an error (step 215).

In one embodiment, red flags task 200 may be available for a user toreturn to, should a user notice an anomaly in later stages ofprocessing. For example, a workflow may include a red flags tab that mayreturn a user to red flags task 200. Red flags task 200 may then presenta user with each anomaly for review. If the user invalidates a series atthis point, the red flags task 200 may note the series as rejected andclose the workstation.

In one embodiment, red flags task 200 may include a progress baroverlaying a task name. The progress bar may only be visible when a useris presently undergoing the red flags task 200. Each item that must bereviewed may take up an equal portion of the task bar. Therefore, if twoseries are selected for processing, items for a single series may fillup half of the progress bar. In addition, a UI may include a list ofanomalies. The list may be selectable by users and selections (of lackof selections) may be changed if anomalies are detected at a laterstage.

FIG. 3A is a block diagram of an exemplary aorta task 300, according toan exemplary embodiment of the present disclosure. Aorta task 300 may beperformed by server systems 106, based on information, images, and datareceived from physicians 102 and/or third party providers 104 overelectronic network 100. In one embodiment, aorta task 300 may lead auser through segmenting an aorta. In a further embodiment, aorta task300 may proceed towards the next task only after a user accepts someaorta segmentation. In one embodiment, aorta task 300 may furtherinclude calculating an aorta centerline. The aorta centerline may beused in a “Finalize” sub-task to calculate aorta inlet and outletboundary condition planes. In one embodiment, aorta task 300 may includethree modes: inspect, edit, and create. Each mode may have a set ofviews and tools to assist in the inspection, modification, and/orcreation of the aorta segmentation. In one embodiment, aorta task 300may be divided into two components: gross error identification andsection analysis. Gross error identification may allow a user to lookgenerally at the structure of the entire aorta, while section analysismay include more detailed analysis. In one embodiment, step 301 mayinclude viewing and assessing the entire aorta. For example, step 301may include presenting a user with the gross structure of an aorta.Next, step 303 may permit a user to correct the structure of the aorta.In one embodiment, the intention for aorta task 300 may be that a usermay determine the appropriateness of the selection of the series forprocessing, but leave the automatic determination unchanged unless aserious error in the calculation has occurred. Various embodiments ofstep 303 may encompass various thresholds for what constitutes a“serious error” that may warrant correction. Selecting to correct astructure of the aorta, may trigger steps 305-311 of section analysis.For example, step 305 may permit a user to view and assess each section.Then, step 307 may permit a user to edit a section. After editing, auser may be prompted to accept the section (step 309). Alternately, auser may continue to edit a section (step 311). In one embodiment,accepting the section may permit the user to return to step 301 ofglobal analysis of the aorta before selecting another part of the aortaimage to correct. In another embodiment, step 309 of accepting thesection may bring an end to the aorta task 300 and permit a user to goonto the next task, the landmarks task 400.

In another embodiment, gross error identification of aorta task 300 mayfurther include the option to create a portion of the aorta (step 313),rather than correcting each section. Step 313 may include using a“paint” function to draw a portion of the aorta, de novo. Step 313 mayoccur where issues with the aorta segmentation are so great, creatingthe segmentation anew may be preferable to editing a previoussegmentation.

FIGS. 3B-3D depict exemplary interfaces by which a user may inspect anaorta, segment the aorta, and calculate an aorta centerline. The usermay selectively Inspect, Edit, and Create aorta segmentation usingexemplary interfaces of FIGS. 3B-3D, respectively. Each mode may includea set of views and tools to assist in the inspection, modification, orcreation of the aorta segmentation. The available tools at this pointmay include “Accept Aorta” and “Reject Aorta.” Selection of “AcceptAorta” may cause the process to move to the next task, landmarks task400. Selection of “Reject Aorta” may enable aorta edit mode (e.g., shownin FIG. 3C). In one embodiment, aorta task 300 may include a progressbar overlaying the task name (e.g., Inspect, Edit, and Create). Forexample, the progress bar may only be visible when a user is undergoingaorta task 300. In a further embodiment, progress for the aorta task 300may be binary: all or nothing. In other words, the progress bar may beempty when the segmentation is not accepted and 100% when the task iscompleted. For such an embodiment of a progress bar, the 100% status mayonly be seen when navigating back to the aorta task 300 from anothertask.

FIG. 3B is a display of an aorta inspect mode 320, according to anexemplary embodiment of the present disclosure. Aorta inspect mode 320may be used to quickly determine whether gross segmentation of the aortagenerated by automatic algorithms is fit to proceed with processing, orwhether the generated aorta requires modification. In one embodiment,the aorta inspect mode 320 may have a layout that includes multipleimages from straightened curved planar reformations (sCPRs) and severalimages from isosurfaces (ISOs). sCPRs may include displays of image dataof an entire length of a tubular structure in a single view. Forexample, sCPRs may include using a straightened representation of thestructure's centerline. A centerline of the structure may be needed togenerate this view. In some cases, the sCPRs may be based on curvedplanar reformations (CPRs). ISOs may include displays of a 3Drepresentation of a signed distance field (SDF) of a currentsegmentation.

In one embodiment, one form of aorta inspect mode 320 may include foursCPRs 321 and four ISOs 323, arranged such that the ISOs 323 are belowthe sCPRS 321. In one embodiment, the sCPRs 321 may be constructed bythe aorta inspect mode 320 using an aorta centerline generated fromautomatic algorithms. An sCPR 321 may be oriented such that a superiordirection is towards the top of a screen. The sCPRs 321 may then berotated to show 0°, 45°, 90°, and 135° planes. The sCPRs 321 may becentered such that the top (first point) of an aorta centerline islocated at the top border of a view. Zoom level for all sCPRs 321 in theaorta inspect mode 320 may remain at the same level. The sCPRs 321 maybe zoomed such that a “radius” of the segmentation (and buffer regions)may be shown in the view. The sCPRs 321 may also be auto-window and/orleveled, wherein aorta intensity may be used to calculate theappropriate window and level.

In one embodiment, the ISOs 323 may also be oriented such that thesuperior direction is situated towards the top of the screen. The ISOs323 may be rotated to show 0°, 90°, 180°, and 270° planes. The ISO 323may be centered such that the centroid of the aorta segmentation is at acenter of the view. In one embodiment, zoom level for all ISOs 323 maybe at the same level. In a further embodiment, the ISO 323 may be zoomedsuch that the radius of the segmentation (and buffer region) may beshown in the view. In one embodiment, navigation may be disabled for allviews because the views for aorta inspect mode 320 may be designed toallow users to evaluate aorta segmentation “at-a-glance,” thus allowinga reproducible binary decision for acceptance. Navigation may beunavailable for aorta inspect mode 320, in one embodiment, to facilitatea reproducible binary decision for acceptance. Fine details may not becritical for aorta segmentation and are therefore not emphasized inaorta inspect mode 320.

In one embodiment, aorta inspect mode 320 may include available toolsAccept Aorta 325 and Reject Aorta 327. A selection of Accept Aorta 325may cause continuation to the next task, landmarks. Selection of RejectAorta 327 may enable aorta edit mode 340.

FIG. 3C is a display of an aorta edit mode 340, according to anexemplary embodiment of the present disclosure. Aorta edit mode 340 maybe used to provide tools, views, and navigation capabilities to assistin a quick and reproducible modification of aorta segmentation. In oneembodiment, aorta edit mode 340 may include one sCPR 341, three imagesfrom multi-planar rendering (MPR) 343 a-343 c (or MPRs 343), and an ISOview 345. In one embodiment, the sCPR 341 may be located at the left ofthe screen as a reference, while the other four views share the rest ofa screen space in a 2×2 configuration, with the ISO 345 in a bottomright corner. The sCPR 341 may be constructed using an aorta centerlinegenerated from automatic algorithms. In one embodiment, the sCPR 341 maybe oriented such that a superior direction is towards the top of thescreen. The sCPR 341 may be centered such that the top (e.g., firstpoint) of the aorta centerline is located at the top border of the view.The sCPR 341 may be zoomed such that the “radius” of the segmentation(plus buffer) may be shown in the view. The sCPR 341 may also beauto-window and/or leveled, wherein aorta intensity may be used tocalculate an appropriate window and level.

In one embodiment, the MPRs 343 may be constrained to move and rotatebased on the aorta centerline. MPR 343 a may include a cross-sectionalview; MPR 343 b may include a lateral view, and MPR 343 c may include alongitudinal view. The lateral MPR 343 b and longitudinal MPR 343 c maybe oriented such that the superior direction is towards the top of thescreen. The cross-sectional MPR 343 a may have an orientation initiallyset to synchronize with the longitudinal MPR 343 c. The MPRs 343 may becentered around an aorta centerline point. In addition, zoom may befixed and kept consistent across all the MPRs 343. In one embodiment,the MPRs 343 may share the same window and/or level as the sCPR 341.

In one embodiment, the ISO 345 may be initially oriented such that thesuperior direction is towards the top of the screen. The ISO 345 may becentered, such that a designated center of aorta segmentation ispositioned at the center of the view. In one embodiment, the ISO 345 maybe zoomed such that the radius of the segmentation (along with thebuffer region) is shown in the view.

In one embodiment, the aorta edit mode 340 may include constrainednavigation. Alternatively or in addition, the aorta edit mode 340 maystem mostly from an Inspector Gadget 347 located in the sCPR 341. Forexample, dragging the Inspector Gadget 347 up and down may re-center andpan all views of MPRs 343 such that centering points remain on the aortacenterline and the center point stays positioned at the center of theview. Similarly, dragging the Inspector Gadget 347 left and right mayrotate all views of MPRs 343 and the sCPR 341. Rotation in thecross-sectional view and longitudinal views may occur around the aortacenterline. Rotation in the lateral view may occur around the transverseaxis. 3D rotation, zooming, and panning may be available on the view ofISO 345.

Further, aorta edit mode 340 may include an exemplary interface by whicha user may edit the aorta segmentation. Specifically, in one embodiment,aorta edit mode 340 may provide two sets of tools for editing: SmartPaint 349 and Nudge 351. While these tools may be used in various formsof functionality for different sections, aorta edit mode 340 may includespecific behaviors for each tool. Smart Paint 349 may include two tools:inside seeds and outside seeds. Use of either of these tools mayautomatically update segmentation. For example, an update may occur inone second or less. An algorithm may use a most recent segmentation anduse the additional seeds to modify the latest segmentation. In oneembodiment, tools associated with Smart Paint 349 may work on MPRs 343.In one embodiment, the tool, Nudge 351, may also work on MPRs 343. Nudge351 may permit users to move a seed a small degree. Updates from usingNudge 351 may be near real-time.

FIG. 3D is a display of an aorta create mode 360, according to anexemplary embodiment of the present disclosure. Aorta create mode 360may be used to provide tools, views, and navigation capabilities toassist in creation of aorta segmentation. The aorta create mode 360 maybe triggered where an algorithm produces a segmentation thatmisidentifies the aorta or produces large errors that would beinconvenient to resolve in the myocardium edit mode 540 described inFIG. 5C. In one embodiment, the aorta create mode 360 may include threeMPRs 361 a-361 c (or MPRs 361) and an ISO 363 arranged in a 2×2 layout.In one embodiment, the MPRs 361 a, 361 b, and 361 c may initially showthe transverse, coronal, and sagittal planes, respectively. The MPRs 361may be centered at the center of the volume of the image shown. The MPRs361 may be set to a default zoom such that the image volume fills one ofthe views. In one embodiment, the three MPRs 361 may have the samewindow and/or level, set to an automatic preset. In this embodiment, thethree MPRs 361 may share a centering point, zoom, and wind and/or level.Rotation may be synchronized between the three MPRs 361 such that theplanes being displayed at any time are always orthogonal.

In one embodiment, the ISO 363 may initially be blank when enteringaorta create mode 360, but the ISO 363 may populate as an aortasegmentation is generated. The ISO 363 may be oriented such that thesuperior direction is towards the top of the screen. In one embodiment,the ISO 363 may be centered such that the “center” of the aortasegmentation is at the center of the view. The ISO 363 is zoomed suchthat the radius of the segmentation (including a buffer region) is shownin the view.

In one embodiment, the aorta create mode 360 may include fullnavigation, meaning users may rotate, pan, re-center, reset orientation,zoom, and adjust a window and/or level on any MPR 361. In oneembodiment, aorta create mode 360 may include the tool, Smart Paint 365.Segmentation may occur when a user selects a button, Update Segmentation367. In one embodiment, segmentation may re-calculate based on seedsfrom Smart Paint 365. For example, if segmentation is based only onseeds from Smart Paint 365, aorta create mode 360 may discard theinitial segmentation. The update may therefore take longer than SmartPaint 349 of the aorta edit mode 340. In one embodiment, Smart Paint 365and Update Segmentation 367 may only work on the MPRs 361.

In combination, the aorta inspect mode 320, aorta edit mode 340, andaorta create mode 360 may be used in the following manner. When theaorta task 300 is enabled, a user may use sCPRs to view segmentationresults. At this stage, the user may be watching for whether aortasegmentation is high enough up the ascending aorta. The user may alsolook for whether a valve point is sufficiently covered by thesegmentation. Further, a user may observe whether ostia are visible insCPRs. In one embodiment, volume rendering (VR) images may be used tosupport the decisions. For example, VR images may display a 3Dprojection of a 3D data set. One such case may include a VR displaymapping intensity to opacity and color, using a transfer function. If anaorta is deemed acceptable, a user may select “Accept Aorta” (e.g.,Accept Aorta 325 of FIG. 3B) and move on to the landmarks task 400.

If the user desires clarification or if the user observes a problem inthe segmentation, the user may enable the aorta edit mode 340 byselecting “Reject Aorta” (e.g., Reject Aorta 327 of FIG. 3B) to triggerthe aorta edit mode 340, associated with steps 307 and 311 of aorta task300. In the aorta edit mode 340, a user may navigate on the sCPR 341 orrotate the MPRs 343 to view segmentation and again evaluate if thesegmentation is complete. In one embodiment, Smart Paint 349 may be usedto first modify segmentation as close as possible to desiredsegmentation. Edits using Smart Paint 349 may affect a local section ofthe aorta segmentation, and using both inside and outside seeds may helpdefine a better segmentation. Smart paint 349 may be used to make largeredits, while minor edits to segmentation may be completed using Nudge351. Both Smart Paint 349 and Nudge 351 tools may operate in a threedimensional context, so a user may be reminded to scroll through imageslices to check the effect of the tool. In one embodiment, a user may beprompted to verify ostial segmentation. For the ostial segmentation, theuser may again, ask whether aorta segmentation is high enough up theascending aorta, whether a valve point is included, etc. After edits arecomplete, a user may click “Accept Aorta,” including Accept Aorta 325 ofFIG. 3B.

In one embodiment, aorta create mode 360 may be the preferred mode for ascenario where edits to a segmentation cannot achieve a desiredsegmentation. For example, aorta create mode 360 may cause a discardingof the segmentation to clear a current segmentation and re-center imageson the aorta (e.g., near the valve point). In one embodiment, a user mayrotate a view so that the ascending aorta, valve point, and parts of theostia are visible. A user may add inside and outside seeds to the viewsuch that the extent of the aorta is covered by the inside seeds and theaorta is surrounded by outside seeds. In one embodiment, a user mayselect to rotate to a view so that it is almost orthogonal to a view andthe same extent of aorta may be seen. The rotation may be done on yetanother view, as the views may be constrained to be orthogonal. Again,inside and outside seeds may be added using the criteria above and thesegmentation may be updated based on the new seeds. In one embodiment,these steps may be repeated as needed, using both inside and outsideseeds such that all major features of the aorta are captured. Whensegmentation is complete, a user may return to aorta edit mode 340, ifdesired, to use the centerline from segmentation for navigation. In oneembodiment, users may only return to aorta edit mode 340 to verifysegmentation. Further edits may be discouraged at this point, as thesegmentation may change in unexpected ways due to the discarding oforiginal seeds in the aorta create mode 360. Where no further edits aredesired, a user may select a button, for example, “Accept Aorta,” andmove onto the landmarks task 400.

In one embodiment, the aorta inspect mode 320 may be the initial modewhen users open the workstation. Upon selecting “Accept Aorta,” forexample, a user may move to the next task. All aorta modes may includesome form of an option for “Accept Aorta.” As discussed earlier, aortainspect mode 320 may only have two options: Accept Aorta and RejectAorta. Selecting to eject an aorta may enable aorta edit mode 340. Tomove out of the aorta edit mode 340, users may select “Accept Aorta.” Ifthe aorta segmentation is not easily modifiable, users may select someform of “Discard Segmentation” to completely discard an existing aortasegmentation and activate the aorta create mode 360. In such a case,initial segmentation of the pipeline of images may not be reused.Segmentation may be recalculated according to creation or modificationof segmentation. In one embodiment, users may also move on to a nexttask in the aorta create mode 360 by selecting a form of “Accept Aorta.”If users wish to return to aorta edit mode 340 (e.g., to verify acreated segmentation using the views and layout available in the aortaedit mode 340), users may select a button that may entail, for example,“Enable Edit Mode.”

Regarding background calculations, any changes to aorta segmentation mayimpact downstream tasks. In aorta edit mode 340, Smart Paint 349 orNudge 351 may cause segmentation to exclude an ostium candidate oraccepted ostium. In such a situation, the ostium point may be deleted.If the candidate was an accepted ostium, a warning message may appearreminding the user of the status of the ostium point. In one embodiment,an aorta centerline may not be recalculated if edits are done in theaorta edit mode 340.

In one embodiment, selecting aorta create mode 360 may automaticallydelete all previous calculations, and the segmentation may berecalculated with every update in segmentation made in aorta create mode360. Upon acceptance of an aorta segmentation in aorta create mode 360,an aorta centerline may be recalculated and used for the next task(e.g., landmarks task 400) and for boundary condition generation.

In one embodiment, users may select to return and/or re-enable the aortatask 300 from any other task. In one embodiment, to return to any of theaorta tasks 320-360 from any other task, an initial mode for display maybe the aorta edit mode 340. In addition, any ostia accepted fromlandmarks task 400 may be visible on all views and any accepted ostialsegmentation from lumen task 700 may be visible on all views. In oneembodiment, the option to “Accept Aorta” may remain inaccessible untileither edit or create tools are used. In other words, the aorta task 300may ensure that a user at least reviews images of an aorta beforeaccepting it.

FIG. 4A is a block diagram of an exemplary landmarks task 400, accordingto an exemplary embodiment of the present disclosure. Landmarks task 400may be performed by server systems 106, based on information, images,and data received from physicians 102 and/or third party providers 104over electronic network 100. In one embodiment, landmarks task 400 maybe used to review and/or modify ostium points and the valve point. Forexample, landmarks task 400 may lead a user through identifying allcorrect ostia points, and then through identifying the correct locationof the aorta valve point. The ostium points and valve point may serve tostandardize origination of vessel centerlines and aorta trim planes,respectively. In one embodiment, the landmarks task 400 may be similarto the aorta task 300 in that a user may not proceed forward in the caseunless at least ostium is identified.

In one embodiment, landmarks task 400 may have, essentially, only onemode (further described in FIG. 4B), although review and modification ofostium points and the valve point may occur sequentially. For example,landmarks task 400 may include viewing and assessing each candidate(i.e. each possible ostium) (step 401). Users may then be prompted todetermine whether a candidate ostium is suitable (step 403). In oneembodiment, a user may accept a candidate, thus verifying or “accepting”an ostium candidate (step 405). In one embodiment, accepting the ostiumcandidate may lead to actively identifying an ostium (step 407). In oneembodiment where a user decides at step 403 that an ostium candidate isnot suitable, a workflow may proceed to step 409 of determining whetherall candidate ostium have been visited. If all candidates have beenviewed and assessed, a user may be prompted to verify and activelyidentify the ostium (step 407). If not, the workflow may return to step401, where a user is presented with another ostium candidate.

Once the ostia are identified, a workflow may proceed to step 411 ofsearching for ostia (e.g., imaging false-negative or “FN” ostia). Forexample, a user may be prompted to view and assess a candidate valvepoint (step 413). If the candidate valve point is deemed suitable (step415), the candidate valve point may be designated as an acceptedcandidate and confirmed valve point (step 417). If the valve point isnot deemed suitable at step 415, a user may be prompted to instead,identify a valve point (step 419).

FIG. 4B is a display of landmarks mode 420, according to an exemplaryembodiment of the present disclosure. As previously discussed, landmarkstask 400 may not have separate modes. While review and modification ofostium points and the valve point may occur sequentially, bothactivities may occur using the same layout. In one embodiment, landmarksmode 420 may include one sCPR 421, three MPRs 423, and an ISO 425 in aVR. In one embodiment, the sCPR 421 may be located at the left of thescreen, while the four other views may be in a 2×2 configuration, withthe VR in the bottom right corner.

In one embodiment, the sCPR 421 may be constructed using an aortacenterline from the aorta task 300. In one embodiment, the sCPR 421 maybe oriented such that the superior direction is towards the top of thescreen. The sCPR 421 may be centered so that the top (i.e., first point)of the aorta centerline is located at the top border of the view. ThesCPR 421 may be zoomed to a level of detail where the “radius” of thesegmentation (with a buffer region) is shown in the view. The sCPR 421may also be auto-window and/or leveled using aorta intensity tocalculate the appropriate window and level.

In one embodiment, the MPRs 423 may be orthogonal to each other. Uponopening the task, the MPRs 423 may be aligned according to an aortacenterline. In one embodiment, the MPRs 423 may include across-sectional MPR located in the top left, a lateral MPR located inthe top right, and an MPR orthogonal to the two other MPRs, located inthe bottom left of a display. In one embodiment, the lateral MPRs may beoriented such that the superior direction is towards the top of thescreen. The cross-sectional MPR's orientation may be initially set tosynchronize with the longitudinal MPR. The MPRs may be centered aroundan aorta centerline point, and zoom may be the same across all MPRviews. In one embodiment, the MPRs 423 may share the same window/levelas the sCPR 421.

In one embodiment, the ISO 425 may be initially oriented such that thesuperior direction is towards the top of the screen, centered such thatthe ‘center’ of the aorta segmentation is at the center of the view, andzoomed such that the radius of the segmentation (with a buffer region)is shown in the view.

In one embodiment, landmarks task 400 may offer various paradigms fornavigating through the aorta to identify landmarks. In a firstembodiment, navigation may be constrained to lie along an aortacenterline. By using an Inspector Gadget 427 tool in the sCPR 421, auser may rotate and scroll along the aorta centerline. In a secondembodiment, landmarks task 400 may offer full navigation capability,with easy navigation through an ostium candidate list. For example, thesCPR 421 may be rotated, where the Inspector Gadget 427 may synchronizenavigation between a view of the sCPR 421 and views of respective MPRs423. The MPRs 423 may have full rotation, scrolling, zooming, panning,re-centering, and windowing capabilities. In addition, when a candidateis selected from the ostium candidate list, the candidate may bere-centered on all three MPR 423 views. The VR view for ISO 425 mayautomatically re-center and rotate to show a currently selected ostiumcandidate as best as possible. In one case, the VR may have its ownnavigational controls, where the VR may be rotated, zoomed, and panned.

In one embodiment, ostium points may be accepted or rejected using atool, for example, Keep Point 429. In one embodiment, exemplary button,Keep Point 429, may change a status of a selected ostium point to“Accepted.” Exemplary button, Reject Point 431, may change a status of aselected ostium point to “Unaccepted.” In both cases, a user interfaceand colors may update and a next candidate on a list may be selected.

In one embodiment, landmarks mode 420 may further include exemplarybutton, New Ostium Point 433. In one embodiment, New Ostium Point 433may be used to add another candidate to the ostium list. In oneinstance, the new ostium may be added in an Accepted state. In a furtherinstance, an inclusion zone may be displayed upon selection of NewOstium Point 433. For example, a contour of an inclusion zone mayrepresent a space within boundaries of aorta segmentation where anostium point may be placed.

Navigation may further include two tools: Next Point 435 and PreviousPoint 437. In one embodiment, these tools may help navigate to a nextostium candidate and previous ostium candidate, respectively, on anostium list 439. Statuses (e.g. accepted or not accepted) may not beaffected by using Next Point 435 and/or Previous Point 437. After allostia have been reviewed and the ostium portion of landmarks task 400 iscomplete, exemplary button, Accept Ostia Points 441, may update thelandmarks mode 420 to review of the valve point. In one embodiment ofreviewing of the valve point, a user may only reposition the valvepoint, for example, using a tool, Reposition Valve Point 443. If thevalve point that is displayed is incorrect, Reposition Valve Point 443may move a valve point to a desired location. Like an ostia, aninclusion zone may be displayed to indicate where in the model a valvepoint may be placed.

When opening landmarks mode 420, a first portion of landmarks task 400may be to review ostia points. In one embodiment, a first ostium may beautomatically selected and centered in the views. A user may review thepositioning of the first ostia to ensure that positioning is not too faroff for centerline generation. If the positioning is acceptable, a usermay opt to accept the ostium point (e.g., using Keep Point 429). A usermay choose to reject an ostium point if the positioning is notacceptable (e.g., using Reject Point 431). After reviewing all ostia inthe ostium list 439, a user may be prompted to review the aortasegmentation again for any missing ostia. If ostia are missing, theostia may be added with New Ostium Point 433. Once all ostia have beenreviewed and/or added, Accept Ostia Points 441 may be selected to moveon to identifying the valve point. Again, the valve point may beautomatically centered in the views. A user may review positioning ofthe valve point to ensure that a correct point is tagged. If the pointis correct, exemplary button, Accept Valve Point 445, may be selected bya user. If the position is incorrect, a user may move the valve pointafter selecting Reposition Valve Point 443. After repositioning thevalve point, a user may subsequently select Accept Valve Point 445.

In one embodiment, Accept Ostia Points 441 may be selected to move fromostia review to valve point review. To move onto the next task,myocardium task 500, a user may select the last selection of landmarksmode 420, Accept Valve Point 445. In one embodiment, both the ostia andvalve point may serve as guiding points for algorithms further along inthe workflow. The ostia may serve as starting points for vesselcenterlines. Thus, if an unaccepted ostium is accepted by a user, acenterlines algorithm may run using merely accepted ostia and possibly,newly placed ostia added by a user. Similarly, if previously acceptedostia are rejected, centerlines corresponding to those ostia may becleared. The valve point may serve as a landmark for an aortacenterline. The aorta centerline may be used later in the workflow. Forexample, the aorta centerline may be used during modeling to accuratelytrim an aorta inlet at a valve plane. Then, upon acceptance of the valvepoint, an aorta centerline may be recomputed.

In one embodiment, a series of settings may be the default, upon a userreturning to landmarks task 400. For example, the first ostium point ofostium list 439 may be selected upon a user re-opening landmarks task400. In one embodiment, Accept Ostia Points 441 may remain inaccessibleuntil an ostium point is added to the list or until a status is changed.After selecting Accept Ostia Points 441, a user may go to the valvepoint section and be prompted to re-accept the valve point to move on toanother task.

In one embodiment, the landmarks mode 420 may have a progress baroverlaying the task name. The progress bar may only be visible duringthe landmarks task 400. In one embodiment, progress for the landmarkstask 400 may first be divided by “sub-tasks” in landmarks task 400. Forexample, identification of ostia may count towards 75% of the bar anddeification of the valve point may count towards 25%. Within the ostiumportion, acceptance or rejection of each ostium may count towards anequal portion of the 75%. Thus, as a user accepts or rejects eachostium, the bar may progressively fill to 75%, in increments based onthe total ostia available for processing. For example, the incrementsmay be equal in size, based on the total available ostia. The bar mayalso fill just as a visual guideline, where the increment size may notdirectly correspond to and acceptance or rejection of each ostium. Inone embodiment, if an ostium point is added, the percentage of eachostium may be shifted so that a visual portion of the progress bar isequal, but will add to the total length of the progress bar (since theostium is added in an accepted state). After a user accepts all ostia,the bar may be at 75% completion. After accepting the valve point, thebar may reach 100% completion.

In one embodiment, the Inspector Gadget 427 may be displayed in thelandmarks mode 420 on the sCPR 421. In one embodiment, a dot 447 mayindicate rotation of the sCPR 421 and serve as indication to a user thatrotation of the MPRs 423 may be controlled by rotation on the sCPR 421.In addition, positioning of line 449 along a centerline may indicate acentering position of the views of the MPRs 423. Since the landmarksmode 420 may have at least two different methods of navigation,synchronization of the views with the centering point may only berepresented accurately when navigation using the Inspector Gadget 427 isused. If further navigation is used on the MPRs 423, the MPRs 423 may beoriented such that Inspector Gadget 427 may not be displaying thecentering position.

In one embodiment, the sCPR 421 may include a display of a relativeposition of a selected ostium point or valve point. For example, a solidline 451 across the sCPR 421 may indicate a projected location along thecenterline of the point. In one embodiment, line 451 may serve as areference so that the Inspector Gadget 427 may be used to locate aposition of the ostium point or valve point accurately. Accuratelylocating the ostium point and/or valve point may permit the MPRs 423 tobe displayed accurately.

FIG. 5A is a block diagram of an exemplary myocardium task 500,according to an exemplary embodiment of the present disclosure.Myocardium task 500 may be performed by server systems 106, based oninformation, images, and data received from physicians 102 and/or thirdparty providers 104 over electronic network 100. In one embodiment,myocardium task 500 may be used to segment the left ventricle. Forexample, the left ventricular mass may be used for a computational fluiddynamics (CFD) solution, meaning a CFD solution may be calculated fromthe volume of a left ventricle segmentation. In one embodiment, a seriesof images or annotations used for the left ventricle segmentation may bedifferent from a series used for a lumen. In one analysis, the qualityof each series may be calculated for both lumen processing and leftventricle processing. Since a diastolic phase may be vastly preferredfor segmenting the left ventricle, the diastolic phase may beprioritized in almost all cases for left ventricle segmentation. In oneembodiment, a user may not proceed forward without an accepted leftventricle segmentation with a mass in an acceptable range, similar toprevious tasks (e.g., red flags task 200, aorta task 300, and landmarkstask 400).

To aid in segmenting the left ventricle, myocardium task 500 may includetwo components: gross error identification and closer inspection of eachsection. In one embodiment, myocardium task 500 may be analogous toaorta task 300, where gross structure may be evaluated first, then fineranalysis may be performed for each section. In one embodiment,myocardium task 500 may begin with step 501 of gross erroridentification, where a user may be prompted to view and assess an leftventricle, basically in its entirety. The user may be prompted todetermine whether the gross structure is correct (step 503). If a userselects that the structure is generally correct, the user may then bepresented with more detailed analysis, including step 505 for viewingand assessing various sections of the left ventricle. In one embodiment,a user may choose to edit the section (step 507). Subsequently, a usermay either accept the section (step 509) or further edit the section(step 511). In one embodiment where a user may determine after viewingthe gross structure of the left ventricle (step 501) that the structureis faulty, a user may select to create de novo a new model of the leftventricle (step 513). In one embodiment, step 513 may be completed usinga tool, for example, Smart Paint.

In one embodiment, myocardium task 500 may provide tools, views, andnavigation capabilities to assist in review and modification ofmyocardium segmentation. Specifically, the myocardium task 500 may becomprised of three modes: myocardium inspect mode 520, myocardium editmode 540, and myocardium create mode 560. In one embodiment, each modemay include different tools and views for reviewing, editing, andcreating the myocardium segmentation.

FIG. 5B is a display of myocardium inspect mode 520, according to anexemplary embodiment of the present disclosure. In one embodiment, themyocardium inspect mode 520 may permit a user to quickly determinewhether gross structure and segmentation of a myocardium generated by anautomatic algorithms is sufficient to proceed, or requires modification.In one embodiment, myocardium inspect mode 520 may include four longaxis MPRs 521 and nine axial MPRs 523. In one embodiment, the long axisMPRs 521 may occupy a left side of a screen, while axial MPRs 523 may bearranged on the right. The views may be constructed from a long axis ofthe myocardium segmentation, which may be determined by apical and basalpoints.

In one embodiment, long axis MPRs 521 may be constructed using themyocardium long axis, by taking slices at 0°, 45°, 90°, and 135°. Theviews may be oriented such that the apex of the heart is towards theright side of the screen, with the base towards the left of the screen.Segmentation may be centered in the views, such that the long axis isalong the horizontal axis of a view. The views may be zoomed such that asegmentation's closest point to a window boundary may be some bufferaway from the boundary.

In one embodiment, the axial MPRs 523 may be constructed by taking ninecross-sectional slices along the long axis of the myocardium. The slicesmay be equally spaced and span the entirety of the long axis. In oneembodiment, apical and basal slices may be placed a small distance fromthe apex and base so that the top and bottom slices do not show onlyspecks of segmentation. In one embodiment, the MPRs 521 and 523 may beoriented such that the 0° direction is towards the top of the view. TheMPRs 521 and 523 may be centered such that the long axis is at thecenter of the view. The views may all share the same zoom, where thezoom of all MPRs 521 and 523 may be based on the zoom of the slice withthe largest segmentation. In one embodiment, that slice with the largestsegmentation may be zoomed such that the segmentation's closest point tothe window boundary is some buffer away from the boundary.

In one embodiment, navigation may be disabled for views in themyocardium inspect mode 520. The initial set-up of the views may supporta quick decision regarding correctness of the segmentation, sonavigation may be reserved for other modes of myocardium task 500.Myocardium inspect mode 520 may just be intended for gross erroridentification, so in one embodiment, myocardium inspect mode 520 mayinclude only tools to accept the segmentation (e.g., a tool, Accept 525)or reject the segmentation (e.g., a tool, Reject 527). If Accept 525 isselected, the myocardium task 500 may be complete and centerlines task600 may commence. If Reject 527 is selected, the myocardium inspect mode520 may still be seen as complete, but myocardium edit mode 540 may beenabled. With myocardium edit mode 540, a user may correct thesegmentation that caused him to reject the segmentation during themyocardium inspect mode 520.

FIG. 5C is a display of myocardium edit mode 540, according to anexemplary embodiment of the present disclosure. In one embodiment, themyocardium edit mode 540 may permit quick and confident edits tomyocardium segmentation so that accurate and reproducible myocardiumsegmentation may be obtained. In one embodiment, the myocardium editmode 540 may include two MPRs 541 a and 541 b and one ISO/VR 543. In oneembodiment, MPR 541 a may be a long axis MPR that occupies the left halfof the myocardium edit mode 540. MPR 541 b may be an axial view MPR thatoccupies the top right half of the display myocardium edit mode 540. TheISO/VR 543 may occupy the bottom right half of the view.

In one embodiment, the log axis MPR 541 a may be oriented to display theapex of the heart at the bottom of the view and the base at the top ofthe view. In one embodiment, both MPR 541 a and 541 b may have theirzoom level based on the size of contours in the view. The long axis MPR541 a may zoom the view so that the nearest segmentation contour hassome buffer away from the border of the view. The axial MPR 541 b mayhave a zoom such that the largest contour also has some buffer away fromthe border of the view.

In one embodiment, navigation in the myocardium edit mode 540 may bebased primarily on Inspector Gadget 545. In one embodiment, theInspector Gadget 545 may permit a user to probe a particular axial slicein the myocardium. The Inspector Gadget 545 may further permit a user torotate the long axis MPR 541 a to display the long axis view. The ISO/VR543 may also be rotated to inspect for holes or bumps.

In one embodiment, Smart Paint 547 and Nudge 549 may be available toolsin myocardium edit mode 540. In one embodiment, these tools may havefunctionalities specific to the myocardium edit mode 540. For example,segmentation may automatically update after placing inside or outsideseeds. Due to the nature of the myocardium task 500, myocardium task 500may include an “inner” contour and an “outer” contour. The inner andouter contours may be respected when a user employs Smart Paint 547.Smart Paint 547 in myocardium edit mode 540 may be analogous to SmartPaint 349 in the aorta edit mode 340. In one embodiment, the same codemay be used for Smart paint 349 and Smart Paint 547.

In one embodiment, the tool for Nudge 549 may also have functionalityparticular to the myocardium task 500. For example, due to the inner andouter contour, a user may create disconnections in the segmentation byusing Nudge 549. For example, Nudge 549 may be used to push contours inwhatever direction a user may move his pointer. In one embodiment, acontour may be sensitive to any pointer movement or selection when usingNudge 549. For example, pointer movement may shift the position of thecontour, and a selection may be read at myocardium edit mode 540 aspunching a hole through the segmentation. In such cases, a user may usecaution regarding where to place a start point of the nudge movement, soas to not make unexpected or unintentional alterations to contours orsegmentation.

In one embodiment, myocardium edit mode 540 may further include theability to accept and/or reject segmentation. For example, when alledits have been completed and a user deems segmentation to be correct,the user may select a tool (e.g., Accept 551) to move onto centerlinetask 600. If myocardium edit mode 540 may not be used to completesegmentation in a reasonable amount of time, a user may have the optionto create a new segmentation. To provide a user with these options,myocardium edit mode 540 may include a tool, e.g., Discard Segmentation553, which may automatically bring a user to myocardium create mode 560.

FIG. 5D is a display of myocardium create mode 560, according to anexemplary embodiment of the present disclosure. In one embodiment,myocardium create mode 560 may provide views, tools, and navigation fora user to create a myocardium segmentation. Myocardium create mode 560may be used if a pipeline algorithm produces a segmentation thatmisidentifies the myocardium, produces a large number of errors,produces errors cumbersome to resolve in myocardium edit mode 540, or acombination thereof.

In one embodiment, a standard layout for myocardium create mode 560 mayinclude three MPRs 561 and one ISO/VR 563. In one embodiment, the threeMPRs 561 may show the transverse, coronal, and sagittal planes. The MPRs561 may be centered at the image volume center and initially zoomed to adegree such that the image volume fills at least one of the views. Inone embodiment, the MPRs 561 may all have the same window and/or level,set to an automatic preset. In a further embodiment, the three MPRs 51may share a centering point, zoom, and/or window level. Rotation may besynchronized between the three MPRs such that the plans being displayedat any given time may consistently remain orthogonal.

In one embodiment, the ISO/VR 563 may be initially blank when myocardiumcreate mode 560 opens. The ISO/VR 563 may populate as myocardiumsegmentation is generated. The ISO/VR may be oriented such that thesuperior direction is towards the top of the screen and centered suchthat the “center” of the myocardium segmentation is at the center of theview. The ISO/VR 563 may be zoomed such that the radius of thesegmentation (plus a buffer portion) is shown in the view.

In one embodiment, navigation may be available in myocardium create mode560. For example, users my rotate, pan, re-center, zoom, and adjustwindow and/or levels on any of the MPRs 561. In one embodiment,myocardium create mode 560 may include a tool, Smart Paint 565. Forinstance, updating for the segmentation may occur when a user selects atool, e.g., Update Segmentation 567. In one embodiment, the segmentationmay be recalculated based only on the seeds from Smart Paint 565,meaning the initial segmentation may be discarded. In one embodiment,this update may take longer than Smart Paint 365 from aorta create mode360 because of the recalculating. In one embodiment, the Smart Paint 565may only work on MPRs 561. Myocardium create mode 560 may furtherinclude a tool for mode switching, e.g., Proceed to Edit 569. In oneembodiment, a user may return to myocardium edit mode 540 to reviewsegmentation, where the myocardium long axis may be used to review themyocardium segmentation.

In summary, in one embodiment, myocardium task 500 may begin atmyocardium inspect mode 520. If all the displayed contours of theventricle look accurate to a user, the user may immediately select toaccept the model and move on to centerlines task 600. If the user isunsure or if he wishes to modify the segmentation, the user may select atool to reject the segmentation or select to go to myocardium edit mode540. In myocardium edit mode 540, a user may navigate to relevant slices(e.g., using Inspector Gadget 545) in order to make changes. In oneembodiment, a user may employ Smart Paint 547 to edit the segmentation.When a user completes his edits from Smart Paint 547, a user may useNudge 549 for fine adjustments. If the edit tools Smart Paint 547 andNudge 549 are insufficient to successfully finish the segmentation, auser may elect to reject the segmentation and enter myocardium createmode 560.

In myocardium create mode 560, a user may start with standard views ofthe myocardium, oriented along sagittal, coronal, and transverse planes.The views may be oriented as best as possible so that one view displaysthe axial plane along the perceived myocardium long axis. The other twoviews may automatically align to display two long axis views. Users mayuse a tool (e.g., Smart Paint 565) on two long axis views to define arange of the myocardium. In one embodiment, myocardium create mode 560may further provide guidance to users to take extra caution to includeoutside seeds in the left ventricular pool and to exclude visiblepapillary muscle. A user may add seeds to another view so that bothviews have inside and outside seeds, defining a region of interest (ROI)within which to operate. Segmentation may be updated (e.g. using UpdateSegmentation 567) when a user feels that he has finished editing.Selecting Update Segmentation 567 may prompt myocardium create mode 560to calculate the segmentation. After segmentation is generated, a usermay review the result and verify that no gross errors are present. Iferrors are noted, the errors may be corrected immediately (e.g., usingSmart Paint 565). If no errors are noted, a user may return tomyocardium edit mode 540 (e.g., by selecting Proceed to Edit 569). Inmyocardium edit mode 540, a user may again review the segmentation usingthe views and tools available from myocardium edit mode 540. In oneembodiment, a shape model may automatically be used. In a furtherembodiment, a toggle option may be available to toggle between a shapemodel output and a raw smart paint output.

In terms of switching modes, the myocardium task 500 may includemyocardium inspect mode 520, myocardium edit mode 540, and myocardiumcreate mode 560. A user may move sequentially from myocardium inspectmode 520 to myocardium edit mode 540 and between myocardium edit mode540 and myocardium create mode 560. Additionally, a user may advance toa next task (e.g., centerlines task 600) from either myocardium inspectmode 520 or myocardium edit mode 540.

In one embodiment for myocardium inspect mode 520, a user may select toreject a segmentation and move to myocardium edit mode 540 if thesegmentation appears wrong. In myocardium edit mode 540, a user mayselect to reject editing and completely discard the segmentation ifediting may not create sufficient modification to make an acceptablesegmentation. After discarding the segmentation, a user may proceed tomyocardium create mode 560. As previously discussed, after segmentationis completed in the myocardium create mode 560, a user may elect toreturn to the myocardium edit mode 540 and review the segmentation. Inboth the myocardium inspect mode 520 and myocardium edit mode 540, auser may select to accept the segmentation and move on to thecenterlines task 600 if the myocardium segmentation is correct. In oneembodiment, the myocardium create mode 560 may not offer the option todirectly accept segmentation.

In one embodiment, several background calculations may occur regardingthe myocardium segmentation. First, the myocardium long axis may berecalculated if a user returns to the myocardium edit mode 540 followingthe myocardium create mode 560. In addition, myocardium task 500 mayinclude suggested trimming locations in centerlines task 600, that arebased, at least in part, on detection of myocardial bridging. In oneembodiment, the boundary of the myocardium may be used to determine whena vessel dips into the myocardium.

In one embodiment, if a user returns to the myocardium task 500,myocardium edit mode 540 may be activated as the default screen. Theoption to accept (e.g., using Accept 551) may be inaccessible until edittools or create tools are used on the segmentation. In other words, auser who returns to the myocardium task 500 may make some modificationbefore segmentation may be accepted.

In one embodiment, a progress bar may overlap with a myocardium tab whenmyocardium task 500 is the active task. This progress bar may trackcompletion of the myocardium task 500 and progress to 100% if themyocardium is accepted. For the myocardium inspect mode 520, positioningof the long axis and axial MPRs 523 may be displayed on the long axisMPRs 521. As a user hovers over an axial MPR 523, the location of theslice may be displayed on the long axis MPR 521.

FIG. 6A is a block diagram of an exemplary centerlines task 600,according to an exemplary embodiment of the present disclosure.Centerlines task 600 may be performed by server systems 106, based oninformation, images, and data received from physicians 102 and/or thirdparty providers 104 over electronic network 100. In one embodiment,centerlines task 600 may be used to review a centerline tree, forexample, in predefined centerline lengths (e.g., sections) in apredefined order. In one embodiment, a user may start with the rightcoronary artery (RCA) and review sections in order from proximal todistal parts of a main vessel. The main vessel may be determined basedon automatic calculations and the determination may be updated in thenext task (e.g., centerlines labeling task 640). After section review onthe main vessel, a user may be provided with options to review the mostproximal branch on the main vessel and all branches from the mainvessel, moving from proximal to distal parts of the vessel. Followingthe RCA tree review, a user may review the LAD tree, then the leftcircumflex artery (LCX) tree. In on embodiment, the user may not proceedto the next task without identifying and accepting at least onecenterline from each ostium point.

In other words, centerlines task 600 may be used to generate acenterline tree usable for lumen task 700 (e.g., shown in FIG. 7A). Todo so, the centerlines task 600 may include automatic navigation,sections, tools, and a directed and tracked workflow to permit a user toquickly inspect, add to, delete from, and edit the centerline tree. Inone embodiment, the centerlines task 600 and lumen task 700 may bedesigned for identifying correct vessels, as well as identifying correcttopology. Thus, tools are provided to add to the centerline tree, deletefrom the centerline tree, and to edit the position of the centerlines.

In one embodiment, centerlines task 600 and lumen task 700 may includeworkflows organized into sections. For example, centerlines task 600 andlumen task 700 may employ sections as a means of breaking up the dataand segmentation into manageable portions. Sections may refer toconsecutive lengths of centerline along a single centerline path. Forexample, sections may begin at the first point on the centerline path,and proceed in the proximal-to-distal direction. In some cases, sectionlength may be constant across all centerline paths, except possibly atterminal sections of each path. At the terminal sections, if the sectionis too small, the section may be combined into the previous section.Centerline sections may also be dependent on the order. If a main vesseldefinition is changed, sections may be recalculated and may be updatedfrom the previous definition and ordering. Thus, a single centerline mayhave several sections defined along its length. Main vessels,especially, may include more sections defined along its length due tothe ordering. For this reason, the main vessel may be reviewed first.

In one embodiment, sections may also represent a currently availablepiece of a segmentation that a user may be validating or modifying. Forexample, tools in both centerlines task 600 and lumen task 700 maypertain to local sections. For instance, the tools may be unavailablefor an entire vessel tree or segmentation. This way, a user may have aconsistent experience with tools, and not have to worry about how theuse of the tool will affect the segmentation in unexpected ways.

In one embodiment, sections also have a clear indication of its status.Section status serves as a method for keeping track of work that hasbeen done and for keeping track of work that has to be redone. Forcenterlines task 600, sections may be either “unaccepted” or “accepted.”In one example, all sections may start as “unaccepted,” while awaitingreview by a user. If no changes are needed for the length of centerline,a user may select “Accept” to approve or validate the section. Tool usein an “accepted” section may change the status of the section back to“unaccepted,” where the section may be re-accepted for validation. Forlumen task 700, sections may be “un-reviewed,” “reviewed,” or“accepted.” Sections may start as “un-reviewed.” After a user reviewsthe section, the status may change to “reviewed.” When the section andall adjacent sections are reviewed, the section status may be updated to“accepted.” Tools may affect un-reviewed or reviewed sections, butaccepted sections may remain the same. For both centerlines task 600 andlumen task 700, the user may be prompted to accept all sections prior tonavigating or advancing to a next task.

In one embodiment, sections may be ordered according to vesselhierarchy, and by distance from the ostium point. First, sections may beordered from proximal to distal along the main vessel centerline. Afterthe main vessel, the most proximal branch may be next in order. Branchesmay be ordered according to proximity to the ostium.

Sections may also be recomputed through several different methods. Forexample, in centerlines task 600, any tool that changes the length ofthe centerline may prompt a re-computation of sections. These tools mayinclude: a reposition tool, a trim tool, an extend tool, and a splinetool. In lumen task 700, sections may be recomputed based on changes inthe centerline tree or changes in labeling. If a bifurcation is added,lumen sections may be computed on sections dependent on a recomputedsection. The lumen segmentation may also be recomputed. In some cases, asection containing a newly added bifurcation may be updated to a statusof “Un-reviewed.” If the length of a section changes through use of thereposition tool, lumen sections may be recomputed for a currentcenterline path. In such a case, lumen segmentation may not berecomputed and the status may be changed to the lowest previous sectionstatus. If the centerline is trimmed, the terminal section may becropped. In some embodiments, trimming of the centerline may not promptre-computation of the segmentation, but status of a segmentation may bechanged to “un-reviewed.” If labeling is changed, order of vesselvalidation may be affected. Where ordering changes, sections may berecomputed on the affected vessels. Status may also be updated to thelowest previous section status. In situations where a user may edit alumen to exclude an accepted centerline, a corresponding lumen sectionmay cease to have an option to “accept” the section. To do so, the usermay navigate back to the centerlines task, correct the centerline, andreaccept the centerline section.

In one embodiment, centerlines task 600 may include reviewing eachostium by reviewing each centerline (e.g., in sections). For example,centerlines task 600 may start with determining whether an ostium has atleast one centerline (step 601). If so, a user may move on to inspecteach section (e.g., using steps 603-611). For example, inspecting eachsection may include first, inspecting section occlusions (step 603). Forexample, a user may search imaging false-negative occlusions (e.g., “FNocclusions”) (step 605) and inspect centerline placement (step 607).Afterwards, a user may inspect bifurcation candidates (step 609) andsearch imaging false-negative bifurcations (e.g., “FN bifurcations”)(step 611). In one embodiment, a user may inspect a centerline terminus(step 613) after inspecting a section. If the terminus is identified asa new section (step 615), section analysis (e.g., using steps 603-611)may begin again for the new section. If a new section is not identifiedat step 615, a user may evaluate whether the terminus is major/primaryor minor (step 617). In one embodiment, if no ostium is detected at step601, a user may opt to create a centerline (step 619).

FIG. 6B is a display of centerlines mode 620, according to an exemplaryembodiment of the present disclosure. In one embodiment, centerlinesmode 620 may include six views of angular maximum intensity projections(aMIPs) 621, a view of an MPR 623, a view of a CPR 625, and a view of aVR 627. aMIPs may include views similar in structure to sCPRs in thatthe display may include image data of an entire length of a centerlinein a straightened view. One distinction between an aMIP and a sCPRthough, may include the aMIP being based on an MIP taken around acenterline in a wedge shape of 60°, whereas sCPRs are displays based ona plane. In one embodiment, the aMIPs 621 may be arranged on the leftside of the display, such that they resemble three sCPR views. The MPR623 may take up half of the remaining right, vertical space. The CPR 625and VR 627 may share the remaining space equally. In one embodiment, theaMIPs 621 may display 0°-60°, 180°-240°, 60°-120°, 240°-300°, 120°-180°,and 300°-360° wedges. In one embodiment, the aMIPs 621 may be orientedsuch that the proximal direction is towards the top of the screen, wherethe origin of the vessel centerline may be located at the top of thewindow. In one embodiment, the aMIPs 621 may automatically zoom suchthat vessels of a certain length fill the screen and vessels longer thanthe given length are zoomed out such that the entire vessel is visiblein the aMIPs 621.

In one embodiment, the MPR 623 may be constrained to rotate about aselected centerline. For example, the MPR 623 may be centered,initially, depending on the currently selected section. If a currentsection has bifurcation points in the section, an initial center pointmay be the first bifurcation point in the section. If the section doesnot have bifurcations, the initial center point may be the center of thesection. The MPR 623 may be oriented such that the proximal direction isat the top of the screen, and the MPR 623 may be constructed such thatthe tangent vector of the center point defines the up and downdirections of the view. The initial rotation of the view may also bedetermined by the section. If the centering point is a bifurcation,initial rotation may be one such that the branch is most visible in theMPR 623. If a bifurcation is not selected, rotation may not update. TheMPR 623 may be zoomed such that the section may fill the vertical spaceof the view. In one embodiment, the MPR 623 may have the same windowingas the aMIPs 621.

In one embodiment, the VR 627 may be cropped to the heart mask. The VR627 may automatically rotate to display the analyzed section at a centerof the screen. In one embodiment, the VR 627 may be oriented such thatthe superior direction is towards the top of the screen. The VR 627 maybe zoomed such that a section is visible in the view.

In one embodiment, navigation in the centerlines mode 620 may beaccomplished through interaction with the aMIPs 621 and the MPR 623.Navigation tools for clicking to re-center or probing may be available.In one embodiment, an Inspector Gadget 629 may be an exemplarynavigational tool that provides these capabilities. The Inspector Gadget629 may appear as a colored line when a cursor is detected to behovering over aMIPs 621 and MPR 623, for instance. The line may be anindication of the probe location. In one embodiment, a user may thenclick to re-center a view of the MPR 623 to a centerline indicated byInspector Gadget 629. In one embodiment, the Inspector Gadget 629 may berestricted a selected section and any previously accepted sections onthe selected centerline. In such an embodiment, Inspector Gadget 629 maynot be operable on unaccepted sections or on sections in differentcenterlines. In one embodiment, centerlines task 600 may haveconstrained rotation capabilities. For example, dragging left or righton the aMIPs 621 and/or MPR 623 may enable the constrained rotation. Inone embodiment, the rotation may occur at a selected centering point androtation may be about a tangent to the vessel centerline at the selectedcentering point. In one embodiment, MPR 623 may be scrolled ten slicesabove and below the image plane containing the center point.

In one embodiment, tools available in centerlines mode 620 may beseparated into different portions of the UI, with each portion sharingthe naming scheme of different modes in other tasks. For example,centerlines mode 620 may include an inspect section 631 and edit section633. The inspect section 631 may include an option that may change astatus of a selected section and all bifurcations in the section to“accepted.” For example, exemplary tool Accept Section 635 may have sucha function of changing the status of the sections and respectivebifurcations undergoing analysis. In one embodiment, Accept Section 635may further operate as a navigational tool, where selection of AcceptSection 635 may automatically prompt navigation to the next unacceptedsection. Inspect section 631 may further include tools, for example,Previous Bifurcation 637 and Next Bifurcation 639. These tools may alsobe navigational tools, automatically navigating to the previous or nextbifurcation in the section being processed. Selection of NextBifurcation 639 may also change a status of the currently selectedbifurcation to “accepted.” Another navigational tool may include anoption for navigating to a previous section. In one embodiment, toolsmay also be bound to keyboard shortcuts, rather than or in addition toappearing as buttons on a display.

Edit section 633 may include several tools, for example, RepositionCenterline 641, Add Branch 643, Move Bifurcation 645, Delete Branch 647,Delete Occlusion 649, Trim 651, Extend Centerline 653, Trim and AddOcclusion 655, Reposition Centerline 657, etc. to edit the centerlineposition of selected and distal sections. In one embodiment, a user mayplace control points on MPR 623 to define the location of a centerline.Each newly placed control point may cause a repositioning algorithm torun and update the centerline. In one embodiment, control points mayalso be moved and deleted. For example, a user may hover a cursor over acontrol point until it is highlighted in order to move the controlpoint. A user may then click and drag on the highlighted point to movethe control point. To delete a control point, a user may hover over thepoint until it is highlighted. For example, a user may click to selectthe control point and select Delete Branch 647 or Delete Occlusion 649to delete a selection. Reposition Centerline 657 may also cause arepositioning algorithm to run when moving or deleting a control point.In one embodiment, Add Branch 643 may add a centerline tree from alocation specified on MPR 623 back to a section centerline. Abifurcation added from Add Branch 643 may have a default status of beingunaccepted, although it may be presented for processing. Add Branch 643may connect a centerline from a section being processed, to a placedcontrol point, e.g., creating a straight line path if the algorithmfails to find a likely path.

Move Bifurcation 645 may allow a user to slide a selected bifurcationpoint up and down a centerline. For example, the bifurcation may only beslid proximally to a next proximal bifurcation and/or slid distally to anext distal bifurcation location. In one embodiment, Move Bifurcation645 may be operable only when a bifurcation is selected. Thus, ifInspector Gadget 629 is used to move off of a bifurcation point,Previous 637 and Next Bifurcation 639 may be used to select bifurcationpoints that need to be moved. In one embodiment, use of Move Bifurcation645 may cause the status of selected sections and any sections throughwhich the bifurcation passes, to change to “unaccepted.” Delete Branch647 may remove the centerline tree and bifurcation point of the selectedbranch. Like Move Bifurcation 645, Delete Branch 647 may work only if abifurcation point is selected. Delete Occlusion 649 may remove anocclusion tag for a selected centerline. In one embodiment, DeleteOcclusion 649 may be available only if the selected centerline is taggedas an occlusion. Trim 651 may be used to delete a centerline tree distalto a user-specified point. In other words, all pathlines distal to theplaced trim point may be removed from the tree. If the trip point isused near a bifurcation point, Trim 651 may cause a bifurcation point tohighlight, and Trim 651 may remove the selected pathline, the centerlinetree distal to that point, and the bifurcation point. Extend Centerline653 may create a centerline from a selected centerline terminus to aplaced point. In one embodiment, the tool may be available only in theterminus section of the centerline. Trim and Add Occlusion 655 may beused to trim a selected centerline and add an occlusion tag to thecenterline. Create Spline 659 may be a tool available in a “create”portion of centerlines task 620. Create Spline 659 may create a splineline from a centerline terminus to user-placed points. In oneembodiment, the spline line may continue until a user finalizes thespline by, for example, double-clicking. Create Spline 659 may also onlybe available in the terminus section of the centerline.

In one embodiment, the intended workflow for centerlines mode 620 may befor a user to review each section in the centerline tree for incorrectcenterline placement, incorrect branch identification, incorrect branchplacement, missing branch identification, etc. Reposition Centerline641, Add Branch 643, Move Bifurcation 645, Delete Branch 647, and DeleteOcclusion 649 may be used for these reviews. At a terminus section, auser may also check for whether the vessel is extended far enough andwhether there should be an occlusion tag on the centerline. Trim 651,Extend Centerline 653, and Trim and Add Occlusion 655 may be used forreviews at the terminus section. In one embodiment, a user may check forcorrect centerline placement as a first line because the centerline mayaffect setup of subsequent views. All of the views may be constructedaround the vessel centerline, so the placement of the centerline withinthe vessel may be extremely important.

Next, each branch within the section may be inspected for itscorrectness. The check may include correct identification, as well ascorrect placement. If either is incorrect, a user may use Delete Branch647 and Move Bifurcation 645, respectively, to correct the branch.Finally, centerlines mode 620 may provide capability to inspect formissing branches. For example, a centerline may be added where a branchis seen. If a branch is too small to be reliably included in a model, asuggested trim location may show a location to be near a bifurcationitself. The aMIPs 621 may be used for this activity since branches maybe visible in the views in at least one of the aMIPs 621. If anything isseen in the aMIPs 621, a user may place the Inspector Gadget 629 at alocation and further inspect the location in the MPR 623.

A special case may exist if a centerline exits the lumen. In thissituation, if Reposition Centerline 641 is unable to move the centerlineback into the vessel or if the centerline is entering the wrongstructures, Trim 651 may be used to cut the centerline. Upon reaching acenterline terminus, a user may decide to extend the centerline or trimthe centerline to a more suitable length. A proposed trim location maybe provided by centerlines task 620. If a centerline is too short, auser may employ Extend Centerline 653 to add to a centerline length. Ifa proposed trim location is not yet visible in views, a user maydetermine that the centerline is still too short. Additionally, ifplaque or disease is detected in a vessel, the vessel may be extended.If the vessel is long enough and no plaque is visible, a user may accepta proposed trim location and trim the centerline.

In one embodiment, a user may inspect a centerline tree in a specifiedorder that proceeds vessel to vessel, section by section. The sectionreview may occur from a proximal end to a distal end. For each section,inspection may follow the analysis as described for centerlines task600. After each section is accepted, a user may reach the terminussection of a selected centerline and check the length of the centerline.After accepting the terminus section, a user may select a nextcenterline and repeat the process for that centerline.

The centerlines task 600 may include all the modes in one interface(e.g., centerlines mode 620) rather than separating the interface intodifferent modes, like the aorta task 300 and myocardium task 500.Sections may be used to control the extent of the review. For example,tools in centerlines mode 620 may be available for each section (e.g.,inspect section 631 and edit section 633) and use may be controlled byeach section's properties and process. For example, Create Spline 659may be operable only when a user is reviewing a centerline terminussection.

In one embodiment, a user may move past centerlines mode 620 when allsections are accepted. Upon acceptance of a last section of thecenterline tree, centerlines task 600 may be seen as completed andcenterlines labeling task 640 may be activated.

In one embodiment, the centerlines task 600 may include severalbackground calculations not immediately exposed to a user. For example,one set of calculations may include an automatic calculation ofsections. For example, sections may be recalculated following any toolthat modifies centerline length. In some cases, the recalculation maynot affect the workflow. If a user edits a previously accepted sectionand makes an edit that changes the length of a selected centerline(e.g., an edit, an extension of the centerline, or trimming of thecenterline), the section and any distal sections may be updated andtheir respective status may change to “unaccepted.”

In one embodiment, calculations may further include backgroundcalculations for the ordering of images and sections for centerlineinspection. Centerline inspection may be based on several factors.First, vessels may be ordered by the main vessels (e.g., RCA, LAD, andthen LCX) to better support an intuitive guided workflow and standardizeordering. The labels may be available either from user confirmation orautomatic pipeline results. If the labels are not available, a firstvessel may be selected by length. In one embodiment, the next images maybe in order of branches off the main vessels, in order of proximal todistal. The order after that, may be onto branches of branches, in orderfrom proximal to distal. Thus, all second generation branches off afirst branch may be reviewed before the second branch and its branches.

Yet another calculation may include a proposed trim location algorithm.The algorithm may place a point on the centerline tree indicating asuggested or proposed trim location, based on a radius estimate of thesegmentation. The suggested trim location may be at every centerlineleaf vessel, meaning the last segment of centerline from a bifurcationto a vessel terminus point. Therefore, when a centerline is added to atree, a new suggested trim location may be computed for each newlycreated leaf vessel.

If a user returns to the centerlines task 600, for example, from lumentask 700, a section selected in lumen mode 720 may be selectedautomatically in centerlines mode 620 as well. For all other tasks, thelast visited section in centerlines mode 620 may be the interfacepresented to a user when a user returns to centerlines task 600. In mostcases, the “last visited section” may be a topology tree. In oneembodiment, Accept Section 635 may be unavailable or inaccessible untila tool used on the centerline tree invalidates acceptance of thecenterline section.

In one embodiment, a progress bar may overlap a centerlines tab when thecenterlines task 600 is active. The bar may track the number ofcurrently viewed sections for the entire tree. As more sections areadded, the progress bar may update according to a correct percentage.Similarly, as each section is reviewed and accepted, the bar mayautomatically fill until all sections are reviewed and accepted.

FIG. 6C is a display of centerlines labeling mode 640, according to anexemplary embodiment of the present disclosure. In one embodiment,centerlines labeling mode 640 may include providing a user with userinterfaces where a user may identify main vessel labels. In oneembodiment, labels may be used to determine a correct ordering, perhapsused in the lumen task 700. More importantly, the labels may be used ina final report to report on FFRct values. For example, labels may beidentified in the following order: RCA, LAD, LCX, and Ramus. Theselabels may be used to set vessel-specific dilation factors inpre-solver/solver calculations. In a further embodiment, the labels mayinform labeling on a FFRct report. In one embodiment, a user may reviewall labels and either accept or un-accept the labels in order to proceedto lumen task 700. In other words, such an embodiment may include eachvessel having at least one accepted label before activating lumen task700.

In one embodiment, centerlines labeling mode 640 may include a view thatis a VR 661. The VR 661 may have two available masks for use incropping, e.g., a heart mask and an aorta mask. Each label may have aunique color and/or tag to differentiate and separate between thevessels. In one embodiment, centerlines defined in the centerlines mode620 may be visible in VR 661.

In one embodiment, centerlines labeling task 640 may include fullnavigation controls for VR 661. In one embodiment, a label mayconsistently be selected, so the space bar may be used to enablenavigation. The VR 661 may feature synchronization of a centerline pointwith MPRs in a navigation screen. To synchronize the centerline pointbetween images, a user may double-click on a centerline point on whichre-centering may occur.

In one embodiment, centerlines labeling task 640 may include a labellist 663. The label list 663 may be used to navigate to a selectedlabel. When a label is selected from the label list 663, a label mayhighlight and the VR 661 may rotate to the best angle to display thecenterline with the selected label displayed.

Tools available in centerlines labeling mode 640 may include tools forlabel updates and changing label status. To update a centerline labelwith a new centerline, a user may select a centerline from VR 661. Theselected centerline may update with the selected label. To accept alabel as correct, a tool, Accept Label 665 may be available. If a labeldoes not exist (e.g., a patient does not have an RCA), an exemplarytool, Clear Label, may be provided. In both cases, the status of a labelmay change to either an acceptance or clearing the label, respectively,and a next label in the label list 663 may be selected. For example, theLAD label in label list 663 may be selected following a selection of alabel for RCA. The process may be repeated for all labels in the labellist 663. In one embodiment, the labels themselves may also be selected.This functionality may permit a user to quickly navigate between labels.In the case of either automatic or manual selection, the VR 661 mayautomatically rotate to display the vessel centerline. If, at any time,a centerline with an existing label is selected, the centerline mayupdate with a currently selected label. The previous label may becleared and the centerline may have an updated status of “un-reviewed.”

The masks for cropping may further aid in identification of labels. Inone embodiment, a heart isolation mask 667 may be a default mask forcropping. Where cropping from the heart isolation mask 667 cannot beinterpreted, aorta mask 669 may present an alternative cropping mask. Ifneither mask renders satisfactory results, a user may select a point onwhich to re-center the navigation screen view of VR 661. In oneembodiment, a user may select the point by double-clicking on the vesselcenterline. In one embodiment, this option may be a last resort, used ifthe heart isolation mask 667 and aorta mask 669 are both unable toprovide a user with sufficient information to correctly identify vessellabels. In one embodiment, a tool, e.g., Accept All Labels 651 mayappear after all vessel labels are either accepted or cleared. SelectingAccept All Labels 651 may activate the next task, lumen task 700. In oneembodiment, Accept All Labels 651 may remain unavailable if any labelsare not yet reviewed and/or the status of any labels contain anyun-reviewed labels.

In one embodiment, no background calculations may be expected to occurfor the labeling task itself. However, a centerline order may beredefined from an output of the labeling task of centerlines labelingmode 640. Such a reordering of the centerlines may also result in areordering and recalculation of sections. These changes may affect lumentask 700.

When returning to the centerlines labeling mode 640 from lumen task 700,labeling and statuses may be the same as when a user left centerlineslabeling mode 640. A first label may be selected and Accept All Labels651 may be inaccessible until a label is changed or a different vesselis selected. If a user returns to the centerline task 600 and makes anyedit, a centerline labeling algorithm may be rerun. Then, upon moving tocenterlines labeling mode 640, labels may be re-inspected to ensurecorrect labeling.

In one embodiment, a progress bar may overlap a centerline labeling tabwhen the centerlines labeling mode 640 is active. Each label'sacceptance or clearance may count equally toward a total final, meaningeach label may count as 25% of the progress bar. For the centerlineslabeling mode 640 display, a label may be selected automatically, as adefault. Thus, navigation may be constrained to space bar movementbetween the labels. In other words, centerlines labeling mode 640 mayhave an embodiment where a vessel may be absent of a label. In oneembodiment, selection of a centerline may be default behavior for amouse button.

FIG. 7A is a block diagram of an exemplary lumen task 700, according toan exemplary embodiment of the present disclosure. Lumen task 700 may beperformed by server systems 106, based on information, images, and datareceived from physicians 102 and/or third party providers 104 overelectronic network 100. In one embodiment, lumen task 700 may be used tosegment lumen. In one embodiment, lumen task 700 may generate lumensegmentation for boundary condition definition that may be used for CFDcalculations. Like centerlines task 600, a user may start with the RCA,segment the lumen in sections, then proceed with the same steps throughreview of LAD and LCX trees. In one embodiment, ordering of the vesselsmay change from ordering in centerlines task 600, for example, due tochanges made in centerlines labeling mode 640. In one embodiment,sections may be reviewed from a proximal to distal end of a vessel.

In one embodiment, lumen task 700 may include reviewing each ostium andeach centerline of the ostium. Specifically, such a review may occur byreviewing each section. For example, lumen task 700 may start withviewing and assessing each section (step 701). Then, a user may bepresented with an option to edit the section (step 703). In oneembodiment, a user may determine not to edit the section. Then, a usermay elect to accept a section (step 705). Alternately, a user may enteroptions to edit a segmentation (step 707), after which the lumen task700 may start again at step 701 for a user to view and assess thesection and see whether the section may use editing.

FIG. 7B is a display of lumen mode 720, according to an exemplaryembodiment of the present disclosure. In one embodiment, lumen mode 720may include an sCPR 721, three MPRs 723, and an ISO 725. In oneembodiment, the sCPR 721 may be located on the left of the window, withthe MPRs 723 and ISO 725 dividing the remaining space in a 2×2arrangement. In one embodiment, the ISO 725 may be at the bottom right.

In one embodiment, the sCPR 721 may be oriented such that the proximaldirection is at the top of the screen and the distal direction at thebottom. The zoom may be fixed, but may adapt according to a length ofthe centerline. The one MPR of MPRs 723 may display a longitudinal viewof the vessel, where the view may be centered at a current centerlinepoint. The view may be oriented to have a proximal direction of atangent to the centerline towards the top of the screen. The zoom mayalso be fixed, but show the entire section. Another MPR of MPRs 723 mayshow a cross-sectional view or a view of a vessel orthogonal to thecenterline. The centering point may be the centerline point, where thezoom may be fixed. Yet another MPR of MPRs 723 may show a lateral view,meaning a view parallel to the z-axis of the dataset. The view may becentered at the centerline point for the views and the view may beoriented such that the anterior direction is towards the top of thescreen. The zoom may also be fixed. In one embodiment, the ISO 725 mayshow the isosurface of a section. The ISO 725 may maintain the samecentering, orientation, rotation, and zoom as the top left MPR of MPRs723.

In one embodiment, a primary navigation tool may include InspectorGadget 727. In one embodiment, the Inspector Gadget 727 may be activeonly in a selected section. The MPRs 723 and sCPR 721 may rotate whenthe Inspector Gadget 727 is rotated and the MPRs 723 may update to thespecified centering point when a re-centering operation is performedusing the Inspector Gadget 727. In one instance, all three MPRs 723 mayact in unison for the rotation and centering. In addition to theInspector Gadget 727, typical constrained navigation may be available onthe MPRs 723. Rotation and centering may be synchronized in all theviews.

In one embodiment, lumen task 700 may include Smart Paint 729, Nudge731, and Tube Mode 733 as exemplary tools. Smart Paint 729 may differfrom “Smart Paint” in other tasks. For example, Smart Paint 729 mayaffect either accepted or unaccepted sections, so Smart Paint 729 mayaffect segmentation across section boundaries. Preferably, lumen task700 may mitigate segmentation disconnects that may arise from sectionboundaries.

In one embodiment, Nudge 731 may also have functionality that isspecific to lumen task 700. Because lumen task 700 may include multiplesets of contours, the Nudge 731 may work on contours of parent sectionsrather than contours of branch sections. This functionality may helpNudge 731 mitigate navigation through previous sections, as well as thevarious sets of contours. Thus, for a particular section with abifurcation, Nudge 731 may not affect the contour of a distal branchsection when a parent section is selected. When a selected section is adistal branch section, however, Nudge 731 may affect a contour of theparent section. In one embodiment, Tube Mode 733 may permit work betweensections. Tube Mode 733 may be a tool used to resolve image artifactsand failures in more semiautomatic algorithms. In one embodiment, lumentask 700 may also feature section-based acceptance of the segmentation.Every section of lumen segmentation may be accepted before finalize task800 is activated. For example, a subsequent section may be presented forreview after a user accepts a section. Once all sections are accepted,finalize task 800 may initiate automatically, in one exemplary workflow.

In one embodiment, lumen task 700 may reorder sections depending on anorder from centerlines labeling mode 640. A first section of the RCAcenterline may be a default as the section reviewed first. For eachsection, a user may determine if the segmentation is correct. SmartPaint 729 may be provided for a user to make minor changes. If Nudge 731is used first, subsequent edits using Smart Paint 729 may overrideprevious edits made by Nudge 731. In one embodiment, lumen task 700 mayinclude different centerline paths with different sets of contours.Separation of the contours may be decided upon to facilitate editing. Inone embodiment, edit tools (e.g., Smart Paint 729 and Nudge 731) mayhelp modify different contours of various hierarchies. In this way, auser may help decrease the chances of returning to a previous section todo detailed work, where the work may be facilitated with a view in laterportions of the workflow. In one embodiment, Smart Paint 729 may be usedfor un-reviewed or reviewed sections. In one embodiment, Nudge 731 maywork on contours of a proximal or connecting branch. This means in afurther embodiment, Nudge 731 may be inoperable for contours of distalsections or of branch vessels. This way, users may use Smart Paint 729and Nudge 731 to modify bifurcation of both contours, on a branchsection. In one embodiment, Smart Paint 729 may modify contours of theparent vessel section and branch section on a parent vessel section, butNudge 731 may affect only the parent vessel section.

In one embodiment, Tube Mode 733 may be used when a major image artifactaffects the way segmentation is calculated using typical tools. TubeMode 733 may be used across sections, where Tube Mode 733 may employ onediameter across various sections. Smoothing may be performed at thesection beginning and end to ensure connectivity in the segmentation.Once a last section is accepted, finalize task 800 may automaticallyactivate and a model may be generated. For the model, individualcontours may be integrated to create a single mesh. To move back to aprevious task, a user may select a tab associated with the respectivetask. In one embodiment, various modes of tasks may be unavailableunless a user goes through a particular mode of the task. For lumen task700, lumen mode 720 may be revisited immediately, especially since theexemplary lumen task 700 includes only one lumen mode 720.

In one embodiment, background calculations may occur after completion oflumen task 700 when a user is moving on to finalize task 800. Forexample, warnings may be computed when the segmentation is beingconverted to a model. In one embodiment, a warning may occur if lumensegmentation is separated from aorta segmentation. In anotherembodiment, a warning may occur if a loop is detected in thesegmentation.

In addition to displaying a centering point in the MPRs 723 andInspector Gadget 727 of the sCPR 721, lumen task 700 may include UIelements. For example, lumen mode 720 may further detect and displaymisregistrations. To better separate out distinctions between sections,a selected section may have a different contour indicator from othersections' contours. As discussed previously, lumen task 700 may includedifferent sets of contours for the parent vessel and adjoining branchvessels. Thus, various contours may be displayed on the MPRs 723 indifferent colors.

FIG. 8A is a block diagram of an exemplary finalize task 800, accordingto an exemplary embodiment of the present disclosure. Finalize task 800may be performed by server systems 106, based on information, images,and data received from physicians 102 and/or third party providers 104over electronic network 100. In one embodiment, finalize task 800 may beused to view the trimmed model, add additional views for a final report,and launch a computational fluid dynamics (CFD) calculation on a server(e.g., on server systems 106). In finalize task 800, a user may view afinal model to be used in CFD calculations, cancel or return to aprevious task, or proceed with a CFD solution. Finalize task 800 mayprovide a last verification for a user to approve a model for CFDsimulations. In one embodiment, the finalize task 800 may further beused to save views and FFRct locations, for example, for a final report.In some instances, finalize task 800 may serve as a critical step toensure reports display acceptable views and FFRct locations. In oneembodiment, a user may enter the finalize task 800 with a series ofsteps 801 for inspecting a model. For example, step 803 may include auser viewing and assessing an entire model. The inspection may includelooking for holes, and/or missing pieces in segmentation. For instance,the inspection may include step 805 of verifying the model structure ascorrect or accurate. If a user detects faults in the model or wishes tocorrect any part of the model, the user may exercise step 807,navigating back to a previous task or user interface mode to make thechanges.

If a user approves of the overall model, he may proceed to set up viewsfor the final report. The views may dictate how information from CFDsolutions or calculations may be presented and/or accessed. In oneembodiment, saved views may serve as a starting point, and the model maybe oriented to display each major centerline. In a further embodiment,pins may be placed on the main centerline, distal to any stenoses. Thepins may serve as markers for locations on the model, at which FFRctcalculations are desired. The pins may further indicate locations of themodel where views are desired. Both the FFRct calculations and views maybe included in a final report. In one embodiment, a user may proceed tosteps 809-815 when he may approve the overall model. In one instance,steps 809-815 may include a process for inspecting each pin. Forexample, step 809 may include viewing and assessing each pin and step811 may include editing the pin. Either step 813 or step 815 ofaccepting the pin or further editing the pin, respectively, may followthe step 811 of editing the pin. In one embodiment, steps 817-823 mayinclude a process for setting up views for a final report. In somecases, the views may include a view displayed for each pin location.Step 817 may include viewing and assessing each view, step 819 mayinclude editing the view, and then steps 821 and 823, respectively, mayinclude either accepting the view or editing the view. When all changesare complete, finalize task 800 may launch the CFD solution and closethe workstation.

FIG. 8B is a display of finalize mode 820, according to an exemplaryembodiment of the present disclosure. In one embodiment, finalize mode820 may depict an exemplary interface by which a user may view the finalmodel used in CFD calculations. Furthermore, finalize mode 820 mayprovide a user with a last step to either cancel and return to aprevious task, or to proceed with the CFD solution. In one embodiment,the finalize mode 820 may feature a trimmed mesh model 825 of the lumenand aorta segmentation generated from previous tasks. In one embodiment,the model 825 may be rotated in any of the axes so views may beadequately set up for a final report. Panning the model may also be anoption provided in the finalize mode 820. In some embodiments, finalizemode 820 may not include some navigational tools, e.g., tools forre-centering.

In one embodiment, the finalize mode 820 may feature tools forgenerating and saving views for a final report, including ways to savean existing orientation as a view, recall saved views, delete savedviews, and overwrite saved views. These tools may include, for instance,Add View 827, Save View 829, and Delete View 831. The finalize mode 820may further include tools for placing FFRct pins (e.g., a tool, Add Pin833). In one embodiment, the finalize task 800 may serve as a last taskfor a workstation, meaning a final check before the model exits ananalyst's workspace. If the model is verified and approved as complete,no final edits are to be made, and all views are set up, a user maylaunch a CFD solution from the finalize mode 820. For example, a usermay use Accept Model 835 to close the workstation and launch the CFDsolution. If a user wishes to return to a previous task, a task bar 837may be used to navigate to a previous task.

Moving from the lumen task 700 to the finalize task 800 may includenumerous background calculations. In one embodiment, one backgroundcalculation may include a union of the multiple disjoint segmentationsto create a single segmentation. A further background calculation mayinclude trimming the segmentation and/or defining boundary conditionplanes, for example, an aorta inlet, aorta outlet, and all vesseloutlets. The aorta inlet and outlet may be defined according tocenterlines from lumen task 700. Outlets may be circularized, at leastto some degree, including the aorta inlet plane. In some embodiments theaorta inlet plane may be more circularized than other outlets. Inaddition to trimming, other background calculations may includecalculating views based on ‘standard’ RAO/LAO orientations. The viewsmay be calculated automatically and/or adjusted by a user. Whether viewsare automatic or user-created, tube angulation may be calculated andsaved. Furthermore, a background calculation may include calculatingFFRml. In some embodiments, model 825 of finalize mode 820 may displaythe calculated FFRml.

In one embodiment, the finalize mode 820 may include several userinterface elements to help a user set up and review a final model. Forexample, pre-canned views with ‘standard’ CT tube orientations may beavailable for display. If a model is rotated and saved, a tubeorientation at that time may be automatically saved to the view.Furthermore, centerline labels may be displayed, to serve as a lastopportunity for a user to check the labels for accuracy orappropriateness, and to ensure that each main vessel is marked with anFFRct pin. Another user interface element may include a progress bar.Acceptance of each saved or prepared view may count as an equal portionof the progress bar, such that as more views are added, the proportionof each view may adjust with a new total. The progress bar for thefinalize task 800 may be visible only in the finalize mode 820.

FIG. 9 is a display of navigate mode 900, according to an exemplaryembodiment of the present disclosure. Using the navigate mode 900, auser may interact with a secondary viewer screen, separate from theactual processing of the lumen and myocardium segmentations. Insituations where the locked down views of the operating visualizationsdo not provide enough information, and full navigation, windowing, or analternate series is needed, the navigate mode 900 may be available toprovide more information for the user. In one embodiment, the navigatemode 900 may permit a user to navigate freely and provide as muchinformation to a user as possible. At the same time, the workstation isintended to provide a guided workflow. Since the navigate mode 900 maycounteract the intent of providing guidance through a guided workflow,the navigate mode 900 may be initially hidden. However, in situationswhere provided views following a guided workflow may not provide enoughinformation to make an accurate segmentation, for example, the navigatemode 900 may be available to help a user view the data.

Since the navigate mode 900 is meant to provide additional informationto a user, tools that affect segmentation may not be provided in thenavigate mode 900. Rather, views and tools may be provided to navigate,measure, and display dates, for instance, of data processed in othermodes. Additionally, since information may be obtained from variousseries, a user may select additional series to display side-by-side withviews to any operate screen MPRs (e.g., MPRs dependent on the sCPR,although if no operate screen MPRs are available, there may be no effecton zoom). In the exemplary navigate mode 900, up to two other additionalseries may be selected for display (e.g., series 903).

In one embodiment, a Copy Operating View tool may be provided on athree-dimensional view in an operating screen. This tool may change there-centering point in MPR views in the navigate mode 900 to the selectedre-centering point in the operating screen, reorient MPR views in thenavigate screen to display coronal, sagittal, and transverse planes, andhave no effect on an existing zoom level in the navigate mode 900. TheCopy Operating View tool on an aMIP view in an operating screen maychange the re-centering point of MPR views in the navigate mode 900 to are-centering point along the centerline, orient the navigate mode 900MPR views orthogonal to an aMIP centerline, and possibly change the zoomlevel of the navigate mode 900 MPR views to any operating screen MPRsdependent on the aMIP.

In one embodiment, the navigate mode 900 may include three MPRs and a VRview arranged in a 2×2 layout. The MPRs may initially show transverse,coronal, and sagittal planes. In addition, the MPRs may be centeredbased on the image volume center. As an initial view, the MPRs may beinitially zoomed such that the image volume fills at least one of theviews. The MPRs may have the same window/level, set to an automaticpreset. The three MPRs may share a centering point, zoom, andwindow/level. Rotation may be synchronized between the three MPRs suchthat the planes being displayed at any time may be kept orthogonal.

In one embodiment, the VR view may display an entire dataset, butcropping presets may exist to display the data cropped with a particularsegmentation. The VR may be orientated such that the superior directionmay be kept towards the top of the screen. The VR may be centered suchthat center of the data is at the center of the view, and the VR may bezoomed such that an entire dataset is displayed.

In one embodiment, full navigation may be available on all views, wherethe MPRs and the VR may include options for re-centering, zooming,panning, rotation, and windowing. The user may also reset theorientation so the MPR views reorient to the transverse, coronal, andsagittal planes while the centering point, zoom, and windowing mayremain the same. The user may also activate a home navigationfunctionality, where the home command may reset an orientation so theMPR views reorient to the transverse, coronal, and sagittal planes,reset the centering point to the center of the dataset, reset the zoomto encompass the entire dataset, and maintain the windowing.

In one embodiment, the navigate mode 900 may also provide a user withoptions to synchronize the screen views to some degree, with theinformation that is displayed in an operating screen. By double-clickingon any view in an operating screen, a corresponding image may bedisplayed in the navigate mode 900. This functionality may be part ofthe functionality of Copy Operating View. For example, if a userdouble-clicks in an MPR view in an operating screen, one MPR in thenavigate mode 900 may change the zoom, re-centering point, andorientation to replicate those currently displayed in the operatingscreen. Copy Operating View on a CPR or sCPR view in the operatingscreen may change the re-centering point of the MPR views in thenavigate mode 900 to the re-centering point along the centerline, orientthe navigate mode 900 MPR views orthogonal to the sCPR centerline, andchange the zoom level of the navigate mode 900 MPR views to anyoperating screen MPRs dependent on the sCPR.

In one embodiment, tools available in the navigate mode 900 may beseparated into several sections, e.g., a series selection section, anavigate section, a section of measuring tools, and a section of overlayoptions. For series selection tools 901, up to three series 903 may beavailable for selection. When a series is selected, data displayed inthe navigate mode 900 may be updated to the selected series. Other viewproperties (like centering point, windowing, zoom level, etc.) may notbe affected by a change in series. In one embodiment, a user may alsoview and select the series for lumen and myocardium processing fromcheckboxes 905 below the series display.

The navigate section may contain several tools for navigating throughthe data. For instance, a tool, Reset Orientation 907, may reorient MPRviews when activated. For example, Reset Orientation 907 may prompt theMPR views to display coronal sagittal, and transverse plans whilepreserving existing zoom, window width, and window level. Whenactivated, Home 909 may orient MPR views to display coronal, sagittal,and transverse planes, reset the zoom to encompass the entire imagevolume, and preserve an existing window width and window level. Whenactivated, a too, Crosshairs 911 may display several crosshairs on MPRviews. The crosshairs may be colored and/or automatically update whennavigation is performed on the MPRs. In some cases, the crosshairs maycontinually be synchronized. When activated, Window Level 913 may adjustwindow width and window level as a user drags a cursor from left toright and top to bottom, respectively. When activated, a Copy OperatingWL 915 may adjust window width and window level of the MPR views in thenavigate mode 900 to the same window width and window level settingsexisting at the moment in the operate screen. In the WL Preset 917, aselection of a preset may adjust windowing of all MPRs in the navigatemode 900 to a particular window/level preset. Selection of a preset mayturn MIP on or off, with a displayed thickness.

In one embodiment, another section may contain various measuring tools.For example, a Caliper 919 may permit a user to place a measuring linein MPR views of a navigate mode 900. When activated, a too, ProfileGraph 921, may allow a user to place a graph displaying Hounsfield unitsalong a line in MPR views in the navigate mode 900. Calipers and profilegraphs may be toggled to switch between the two measuring tools. Whenactivated, Hide All 923 may hide all measurements for a currentlyselected image volume. When activated, Remove All 925 may deletemeasurements for a currently selected image.

Yet another section may include several overlay options for screen viewsin the navigate mode 900. For a 3D view, available options may include“full,” “heart,” and “vessels only.” When activated, Full 927 maydisplay a volume rendering for a selected image volume cropped by aheart isolation mask. When activated, Vessels Only 929 may display avolume rendering for a selected image volume cropped by the coronarysegmentation. For the MPR views, available options may include “off,“LV,” and “vessels.” When activated, Off 931 may display work done inthe myocardium task 500. When activated, Vessels 933 may display workdone in the aorta task 300, centerlines task 600, and/or the lumen task700. The final section in the navigate mode 900 may include actionsperformed for the entire case. The list may be used to undo and redoprevious work (e.g., actions performed in the operating screens). Insome cases, the list may be activated by a too, Operating History 935.In one embodiment, the list may be populated and shared for the entirecase, so even series switches and actions performed on other series maybe saved.

In one embodiment, background calculations for the navigate mode 900 mayinclude calculations performed for the measurement tools. For example,Caliper 919 may lay down a 2D length measurement tool and display thelength of the laid-down line in millimeters. Profile Graph 921 maydisplay the Hounsfield units of the voxels underneath the laid-downline.

In one embodiment, the navigate mode 900 may include several userinterface (UI) elements unique to the navigate mode 900. First, for aseries display, the currently displayed dataset in the navigate mode 900may be indicated by a highlighted selection. In some cases, a seriesdisplayed in the navigate mode 900 may be different from a seriesdisplayed in an operating screen. The series displayed in the operatingscreen may be selected for lumen processing or for myocardiumprocessing, depending on the currently activated task in the operatingscreen. The selected case or series may be indicated by the set ofcheckboxes underneath the series selection set of tools. Additionally,the confidence of each algorithm may be output and displayed next to theselection indicators.

Other embodiments of the invention will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A computer-implemented method of manipulatingimage annotations, the method comprising: receiving an image of anindividual's anatomy; automatically determining, using a processor, oneor more annotations for anatomical features identified in the image ofthe individual's anatomy; determining a dependency or hierarchy betweenat least two of the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy; and generating,based on the dependency or hierarchy, a workflow prompting a user tomanipulate the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy.
 2. The method ofclaim 1, wherein the workflow includes presenting the one or moreannotations for anatomical features identified in the image of theindividual's anatomy in an order of greatest number of dependencies tothe least number of dependencies.
 3. The method of claim 2, furthercomprising: prompting a validation request prior to presenting a nextannotation in the workflow.
 4. The method of claim 1, furthercomprising: prompting a validation request for a subset of the one ormore annotations for anatomical features identified in the image of theindividual's anatomy.
 5. The method of claim 4, further comprising:presenting the subset of the one or more annotations for anatomicalfeatures identified in the image of the individual's anatomy multipletimes or to multiple users.
 6. The method of claim 1, furthercomprising: creating one or more visualizations for each of the one ormore annotations for anatomical features identified in the image of theindividual's anatomy.
 7. The method of claim 6, further comprising:receiving one or more modifications to the one or more annotations foranatomical features identified in the image of the individual's anatomybased on tools associated with the one or more visualizations.
 8. Themethod of claim 7, further comprising: determining one or more dependentannotations associated with the one or more modifications; anddetermining a re-computation of the one or more dependent annotationsbased on the one or more modifications.
 9. A system for manipulatingimage annotations, the system comprising: a data storage device storinginstructions for identifying image acquisition parameters; and aprocessor configured to execute the instructions to perform a methodincluding: receiving an image of an individual's anatomy; automaticallydetermining, using a processor, one or more annotations for anatomicalfeatures identified in the image of the individual's anatomy;determining a dependency or hierarchy between at least two of the one ormore annotations for anatomical features identified in the image of theindividual's anatomy; and generating, based on the dependency orhierarchy, a workflow prompting a user to manipulate the one or moreannotations for anatomical features identified in the image of theindividual's anatomy.
 10. The system of claim 9, wherein the workflowincludes presenting the one or more annotations for anatomical featuresidentified in the image of the individual's anatomy in an order ofgreatest number of dependencies to the least number of dependencies. 11.The system of claim 10, wherein the system is further configured for:prompting a validation request prior to presenting a next annotation inthe workflow.
 12. The system of claim 9, wherein the system is furtherconfigured for: prompting a validation request for a subset of the oneor more annotations for anatomical features identified in the image ofthe individual's anatomy.
 13. The system of claim 12, wherein the systemis further configured for: presenting the subset of the one or moreannotations for anatomical features identified in the image of theindividual's anatomy multiple times or to multiple users.
 14. The systemof claim 9, wherein the system is further configured for: creating oneor more visualizations for each of the one or more annotations foranatomical features identified in the image of the individual's anatomy.15. The system of claim 14, wherein the system is further configuredfor: receiving one or more modifications to the one or more annotationsfor anatomical features identified in the image of the individual'sanatomy based on tools associated with the one or more visualizations.16. The system of claim 15, wherein the system is further configuredfor: determining one or more dependent annotations associated with theone or more modifications; and determining a re-computation of the oneor more dependent annotations based on the one or more modifications.17. A non-transitory computer readable medium for use on a computersystem containing computer-executable programming instructions forperforming a method of manipulating image annotations, the methodcomprising: receiving an image of an individual's anatomy; automaticallydetermining, using a processor, one or more annotations for anatomicalfeatures identified in the image of the individual's anatomy;determining a dependency or hierarchy between at least two of the one ormore annotations for anatomical features identified in the image of theindividual's anatomy; and generating, based on the dependency orhierarchy, a workflow prompting a user to manipulate the one or moreannotations for anatomical features identified in the image of theindividual's anatomy.
 18. The non-transitory computer readable medium ofclaim 17, wherein the workflow includes presenting the one or moreannotations for anatomical features identified in the image of theindividual's anatomy in an order of greatest number of dependencies tothe least number of dependencies.
 19. The non-transitory computerreadable medium of claim 18, the method further comprising: prompting avalidation request prior to presenting a next annotation in theworkflow.
 20. The non-transitory computer readable medium of claim 17,the method further comprising: prompting a validation request for asubset of the one or more annotations for anatomical features identifiedin the image of the individual's anatomy.